If you compile GNU Emacs on a supported architecture with more than 32 bits, you might increase the maximum buffer size. The reason this is needed are so-called “tag bits” which tag each pointer with the corresponding datatype. This limits the size of buffers to the largest integer you can represent in Elisp, whose value is given by the variable most-positive-fixnum.
most-positive-fixnum => 268435455
Thus, the limit is 256MB on this particular system:
(/ 268435456.0 1024 1024) => 256.0
Note how we used floating point arithmetic to compute the result!
Anyway, what follows is some technical background, based on the EmacsFaq and some posting on the newsgroups:
Old versions (i.e., anything before 19.29) of GNU Emacs had problems editing files larger than 8 megabytes. As of version 19.29, the maximum buffer size is at least 2^27-1, or 134,217,727 bytes. On a 64bit system, the limit is pushed to 2^59-1 which is almost a million Terabytes.
David Gillespie explains how this problem crops up; while his numbers are true only for pre-19.29 versions of GNU Emacs, the theory remains the same with current versions.
And here are even more details:
From: EliZaretskii Subject: Re: maximum buffer size Newsgroups: comp.emacs Date: Mon, 21 May 2001 19:32:49 +0300
Emacs needs to track buffer positions in each buffer. The variables which track buffer positions (point and its ilk) are all Lisp integers, which have only 28 bits on a 32-bit machine. This is the core reason for the 128MB limit.
By contrast, the maximum amount of virtual memory available to a process is determined by the OS and the limits it imposes on resources available for each process.
The VM limitations are orthogonal to the possible values representable by an Emacs integer. For example, if each buffer is smaller than 128MB, Emacs should be able to have as many of such buffers as the available VM is capable of supporting. If your system can sustain more than 1.28GB of VM, Emacs should be able to have 10 or more 128-MB buffers.
Subject: Re: large file aware? From: HrvojeNiksic (edited by AlexSchroeder) Date: 13 Oct 1999 15:57:34 +0200 Newsgroups: comp.emacs.xemacs
The actual limit depends on XEmacs version and has to do with the limits on Lisp integers. Traditionally, the largest buffer size was 256M (== 2^28; three tag-bits, one bit for sign). As of XEmacs 21, it has been possible to configure XEmacs --with-minimal-tagbits, which uses perverse tricks to get 31-bit signed integers in Lisp. This bumps maximum buffer size to 2^30 (we still need one bit for sign) == 1G.
If a file is larger than 2G, XEmacs will be unable to address it in memory, even if you have enough physical RAM.
If you really really want to try to edit those humongous files in XEmacs, you should get the latest version and compile it in LP64 environment. That way you’ll get maximum buffer size of 2^62 bytes, and large file support for free.
Good luck with editing those files, by the way.
See VLF for a method of viewing awkwardly large files (larger than physical, virtual, or whatever limit you choose for performance issues) a portion at a time.