![[Home]](https://www.emacswiki.org/images/logo218x38.png)
If you compile GNU Emacs on a supported architecture with more than 32 bits, you might increase the maximum buffer size. The reason this is needed are so-called “tag bits” which tag each pointer with the corresponding datatype. This limits the size of buffers to the largest integer you can represent in Elisp, whose value is given by the variable most-positive-fixnum.
For example:
most-positive-fixnum
=> 536870911Thus, the limit is 512MB on this particular system:
(/ 536870911.0 1024 1024)
=> 512.0Note how we used floating point arithmetic to compute the result!
Anyway, what follows is some technical background, based on the EmacsFaq and some posting on the newsgroups:
Old versions (i.e., anything before 19.29) of GNU Emacs had problems editing files larger than 8 megabytes. This limit was pushed further, first to 128MB (since 19.29), then 256MB, and 512MB (since 23.2). On a 64bit system, the limit is much higher than anything you might reasonably be able to open for the foreseeable future.
David Gillespie explains how this problem crops up; while his numbers are true only for pre-19.29 versions of GNU Emacs, the theory remains the same with current versions.
And here are even more details:
From: EliZaretskii Subject: Re: maximum buffer size Newsgroups: comp.emacs Date: Mon, 21 May 2001 19:32:49 +0300
Emacs needs to track buffer positions in each buffer. The variables which track buffer positions (point and its ilk) are all Lisp integers, which have only 28 bits on a 32-bit machine. This is the core reason for the 128MB limit.
By contrast, the maximum amount of virtual memory available to a process is determined by the OS and the limits it imposes on resources available for each process.
The VM limitations are orthogonal to the possible values representable by an Emacs integer. For example, if each buffer is smaller than 128MB, Emacs should be able to have as many of such buffers as the available VM is capable of supporting. If your system can sustain more than 1.28GB of VM, Emacs should be able to have 10 or more 128-MB buffers.
Subject: Re: large file aware? From: HrvojeNiksic (edited by AlexSchroeder) Date: 13 Oct 1999 15:57:34 +0200 Newsgroups: comp.emacs.xemacs
The actual limit depends on XEmacs version and has to do with the limits on Lisp integers. Traditionally, the largest buffer size was 256M (== 2^28; three tag-bits, one bit for sign). As of XEmacs 21, it has been possible to configure XEmacs --with-minimal-tagbits, which uses perverse tricks to get 31-bit signed integers in Lisp. This bumps maximum buffer size to 2^30 (we still need one bit for sign) == 1G.
If a file is larger than 2G, XEmacs will be unable to address it in memory, even if you have enough physical RAM.
If you really really want to try to edit those humongous files in XEmacs, you should get the latest version and compile it in LP64 environment. That way you’ll get maximum buffer size of 2^62 bytes, and large file support for free.
Good luck with editing those files, by the way.
See VLF for a method of viewing awkwardly large files (larger than physical, virtual, or whatever limit you choose for performance issues) a portion at a time.
Emacs uses a gap buffer data structure for representing buffers, see https://nullprogram.com/blog/2017/09/07/ for a readable explanation. This makes lots of edits within a local area fast, but jumping back and forth across a huge file and editing in each spot can be very slow. It also makes the maximum file size bounded (see above).
Moving to alternative data structures has been suggested, e.g.:
For large files: by moving from a gap buffer to a rope representation for buffers, we can partially use memory-mapped backing storage, and even when we do need private, modifiable memory for editing, we can allocate only when we immediately need and not have to move the gap around through humongous amounts of main memory. Such a system would not only improve our support for humongous files, but would also make a 32-bit Emacs capable of editing files larger than its address space.
(Daniel Colascione on emacs-devel 5 Mar 2018 https://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00104.html )