As explained below on this page, adding threads to Emacs is so difficult that it’s basically impossible. Unfortunately, nobody notified the Emacs maintainers of this, so they went ahead and did it anyway, see Threads in the Elisp manual. 😊
More seriously though, some of the difficulties talked about below are avoided because Emacs doesn’t have actual parallelism (yet), only one thread runs at a time.
A new Emacs – even a GuileEmacs – will have no threading.
Threads are difficult to handle. They are even more difficult to handle when the big collection of elisp available was written without threads.
Threads accessing common variables is just the top of the iceberg. What about point – one point per thread? One current buffer per thread? What about buffer-local variables? One thread per buffer?
If at all, it would seem more reasonable to have a strong separation of threads – as if every thread was a separate process. And then we start talking about specific things to share between threads. There could be a special form to declare variables shared accross threads. There could be a mechanism to allow access to a buffer to another thread. The default would be to disallow this – as if every thread was a separate Emacs. As such, every thread would have to run in a separate frame, because a thread cannot take frame-local or window-local variables in other threads into account. Perhaps there would be a special buffer-local variable that would allow a buffer to be displayed in a frame or window controlled by another thread; in that case it would be clear that the original thread’s window-local and frame-local bindings will have no effect.
The following is from #emacs on irc.freenode.net, which is not logged. It took place on 2006-01-17; times are relative to Australian Eastern Standard Time (+1100).
So the legacy of existing EmacsLisp programms would not (theoretically) be a problem.
Had I been in #emacs at the time, I would have said that running multiple emacs instances is one healthy approach. I like a Gnus, ECB, and ERC instance, myself. RAM is cheap, so why fret about single-threadedness? --ChristopherSmith
Well, for example, I am reading this page because I issued a `magit-diff` command on a branch with many changes and I needed something to do while I waited for it to be possible to use that emacs instance again.
Some of us fret about single-threadedness because we like to have to remember which of the three emacsen we opened a particular file in, last week. By having a single emacs, that particular dilemma is avoided. I have been told that it is difficult, but not impossible, to make a multi-threaded emacs, so I think the project should be undertaken. Perhaps we can have a threaded Emacs Virtual Machine that runs the current non-threaded Emacs as children (I don’t know if that even makes sense) and each of those children would run Gnus or ERC, etc. --e1f
An obvious, but effective solution that I personally employ is too have one emacs process for ‘applications’ and one for editing. ERC, VM, newsticker, w3m, and a terminal run in one instance, while my code and whatnot go in the other. This also has the added benefit of not having to manage (or worry about) a large amount of buffers. --Sturm
But this approach (segregated emacsen) negates the feature of emacs that is most wonderful: the everything-is-a-buffer integration that makes it possible to chop your data around and use it everywhere. If my kill-ring for w3m, Gnus, jabber.el was separate from my org-mode buffers and my code, I’d lose 90% of the reasons for using Emacs in the first place. We should think of emacs as the everything-and-the-kitchen-sink (as the old debian package ran) approach that really exposes the power of computing, which is why in today’s networked world threads seem so lacking in it. --ZacharyHamm
I think Riastradh overstated the cost of using multithreading with dynamic binding. It can be done with a small constant CPU overhead by having an indirection buffer in thread-local storage. In conceptual terms, it’s a big per-thread array of void * variables, one for each symbol. Using a variable becomes more expensive by the cost of exactly one pointer dereference. If you were compiling down to native code, most of the dereferences could be trivially optimised out. The per-thread array would waste some memory (anyone got numbers for how many symbols a typical emacs instance uses?), but the really nasty part happens when you evaluate some code that creates a new symbol. Either you need to stop all your threads to expand their indirection tables, or threads need to be able to cope with accesses off the end of their indirection tables. A nice hacky way to do this would be to put a guard page at the end of each indirection table, and initialise all pointers in the indirection table to point to another guard page. Then you stick in a SEGV handler to do all the horrible patchups when a thread tries to access a symbol it doesn’t have an indirection entry for.
The bigger problem with dynamic binding and multi-threading is that dynamic binding makes local analysis of memory interactions effectively impossible. Which means that automatic parallelisation is impossible. Which means that to take advantage of multi-threading, code has to declare itself multi-threading capable, and then perform explicit locking. Which will have very little performance or interactivity benefit while the rest of the code is still running in a single thread. This implies that even if multi-threading capabilities were added to emacs lisp, almost no-one would actually use it unless they were forced to.
Perl addresses the dynamic binding/multithreading problem by essentially running each thread in a separate memory space, with explicit sharing. This is similar to the thread-per-frame approach described above. It’s useful for Perl, because many existing Perl modules make little use of global variables. I would argue that it’s not that useful for emacs. As a thought experiment, how would speedbar work in a thread-per-frame world?
Intel are talking up a future where CPU frequency progressively drops, while the number of cores increases to compensate. In this scenario, single-threaded emacs is going to get slower and slower. However, I don’t seriously think it will ever drop to the speed it was 15 years ago, which was already perfectly acceptable. --goaty
While I think threading along with transition to a more modern language substrate are important, the idea of running multiple instances might be worth developing a bit. If each instance kept in contact with the others, they could synchronize their kill rings among other things. Commands could be provided to list the union of all buffers open on all instances, and to either jump to the instance (using some Xwindows fu or other) or to migrate that buffer and as much of its environment as feasible to the currently used instance. Mechanisms by which this could be achieved include AsyncEval, DistributedEmacsLisp, or even using ILisp/SLIME to connect to an inferior Emacs REPL. – SpaceBat
Why not going down the route of node.js instead and use a single-threaded approach with consequent support for non-blocking io? Most blocking situations I encounter in emacs come from io (think tramp), not from computation. This could be introduced gradually by providing callback-based variants of io calls and leave it up to packages when they will migrate – StefanPlantikow
See my Elnode for an implementation.
Now Emacs has lexical scope one simple use of threads could be to process a stream in parallel. A stream function is one that delivers values, like popping a list. Threads would be useful attached to streams as you could move a ton of processing into a separate thread.