Vasilij Schneidermannatom egg for Chickentag:https://emacshorrors.com,:/RantsEmacs Horrors2019-04-23T20:09:46ZVasilij Schneidermann
<p>Originally I planned to blog about a fun hack, porting the infamous
<a class="reference external" href="https://github.com/panicsteve/cloud-to-butt">cloud-to-butt</a> browser extension to Emacs. The idea was that whenever
you interact with subprocesses instances of “cloud” would be replaced
with “butt”, I picked <tt class="docutils literal">shell.el</tt> for ease of hacking<a class="footnote-reference" href="#id3" id="id1">[1]</a>. The
following snippet is loosely modeled after
<tt class="docutils literal"><span class="pre">ansi-color-process-output</span></tt>, so pardon any weirdness.</p>
<pre class="code elisp literal-block">
<span class="p">(</span><span class="nb">defun</span> <span class="nv">my-filter-shell-output</span> <span class="p">(</span><span class="nf">string</span><span class="p">)</span>
<span class="p">(</span><span class="k">let</span> <span class="p">((</span><span class="nv">start-marker</span> <span class="nv">comint-last-output-start</span><span class="p">)</span>
<span class="p">(</span><span class="nv">end-marker</span> <span class="p">(</span><span class="nf">process-mark</span> <span class="p">(</span><span class="nf">get-buffer-process</span> <span class="p">(</span><span class="nf">current-buffer</span><span class="p">)))))</span>
<span class="p">(</span><span class="k">save-excursion</span>
<span class="p">(</span><span class="nf">goto-char</span> <span class="nv">start-marker</span><span class="p">)</span>
<span class="p">(</span><span class="k">while</span> <span class="p">(</span><span class="nf">search-forward</span> <span class="s">"cloud"</span> <span class="nv">end-marker</span> <span class="no">t</span><span class="p">)</span>
<span class="p">(</span><span class="nf">replace-match</span> <span class="s">"butt"</span><span class="p">)))))</span>
<span class="p">(</span><span class="nb">with-eval-after-load</span> <span class="ss">'shell</span>
<span class="p">(</span><span class="nv">add-hook</span> <span class="ss">'comint-output-filter-functions</span> <span class="ss">'my-filter-shell-output</span> <span class="no">t</span><span class="p">))</span>
</pre>
<p>The API is somewhat murky. A comint output filter function receives a
string argument and is expected to modify the buffer. There’s no
documentation on how to retrieve the positions of the last output, so
I did whatever aforementioned exemplary function does and restrict the
search and replace operations to two markers. How could this possibly
go wrong? See for yourself in the following test session:</p>
<pre class="code shell-session literal-block">
<span class="gp">[wasa@box ~]$</span> <span class="nb">echo</span> cloud
<span class="go">echo butt
butt
</span><span class="gp">[wasa@box ~]$</span> <span class="nb">echo</span> butt
<span class="go">butt
</span><span class="gp">[wasa@box ~]$</span> <span class="nb">echo</span> ponies
<span class="go">ponies</span>
</pre>
<p>Something is definitely wrong here, an extra line is printed if and
only if the replacement would have happened. Most curiously, it
doesn’t mirror the user input, but has the replacement as well. After
debugging this a bit<a class="footnote-reference" href="#id4" id="id2">[2]</a> I remembered that long time ago I’ve set
<tt class="docutils literal"><span class="pre">comint-process-echoes</span></tt> because <tt class="docutils literal"><span class="pre">M-x</span> shell</tt> kept printing the user
input after sending it to the shell. Time to gaze into the abyss:</p>
<pre class="code elisp literal-block">
<span class="c1">;; Optionally delete echoed input (after checking it).</span>
<span class="p">(</span><span class="nb">when</span> <span class="p">(</span><span class="k">and</span> <span class="nv">comint-process-echoes</span> <span class="p">(</span><span class="nv">not</span> <span class="nv">artificial</span><span class="p">))</span>
<span class="p">(</span><span class="k">let</span> <span class="p">((</span><span class="nv">echo-len</span> <span class="p">(</span><span class="nf">-</span> <span class="nv">comint-last-input-end</span>
<span class="nv">comint-last-input-start</span><span class="p">)))</span>
<span class="c1">;; Wait for all input to be echoed:</span>
<span class="p">(</span><span class="k">while</span> <span class="p">(</span><span class="k">and</span> <span class="p">(</span><span class="nf">></span> <span class="p">(</span><span class="nf">+</span> <span class="nv">comint-last-input-end</span> <span class="nv">echo-len</span><span class="p">)</span>
<span class="p">(</span><span class="nf">point-max</span><span class="p">))</span>
<span class="p">(</span><span class="nf">accept-process-output</span> <span class="nv">proc</span><span class="p">)</span>
<span class="p">(</span><span class="nv">zerop</span>
<span class="p">(</span><span class="nf">compare-buffer-substrings</span>
<span class="no">nil</span> <span class="nv">comint-last-input-start</span>
<span class="p">(</span><span class="nf">-</span> <span class="p">(</span><span class="nf">point-max</span><span class="p">)</span> <span class="nv">echo-len</span><span class="p">)</span>
<span class="c1">;; Above difference is equivalent to</span>
<span class="c1">;; (+ comint-last-input-start</span>
<span class="c1">;; (- (point-max) comint-last-input-end))</span>
<span class="no">nil</span> <span class="nv">comint-last-input-end</span> <span class="p">(</span><span class="nf">point-max</span><span class="p">)))))</span>
<span class="p">(</span><span class="k">if</span> <span class="p">(</span><span class="k">and</span>
<span class="p">(</span><span class="nf"><=</span> <span class="p">(</span><span class="nf">+</span> <span class="nv">comint-last-input-end</span> <span class="nv">echo-len</span><span class="p">)</span>
<span class="p">(</span><span class="nf">point-max</span><span class="p">))</span>
<span class="p">(</span><span class="nv">zerop</span>
<span class="p">(</span><span class="nf">compare-buffer-substrings</span>
<span class="no">nil</span> <span class="nv">comint-last-input-start</span> <span class="nv">comint-last-input-end</span>
<span class="no">nil</span> <span class="nv">comint-last-input-end</span>
<span class="p">(</span><span class="nf">+</span> <span class="nv">comint-last-input-end</span> <span class="nv">echo-len</span><span class="p">))))</span>
<span class="c1">;; Certain parts of the text to be deleted may have</span>
<span class="c1">;; been mistaken for prompts. We have to prevent</span>
<span class="c1">;; problems when `comint-prompt-read-only' is non-nil.</span>
<span class="p">(</span><span class="k">let</span> <span class="p">((</span><span class="nv">inhibit-read-only</span> <span class="no">t</span><span class="p">))</span>
<span class="p">(</span><span class="nf">delete-region</span> <span class="nv">comint-last-input-end</span>
<span class="p">(</span><span class="nf">+</span> <span class="nv">comint-last-input-end</span> <span class="nv">echo-len</span><span class="p">))</span>
<span class="p">(</span><span class="nb">when</span> <span class="nv">comint-prompt-read-only</span>
<span class="p">(</span><span class="k">save-excursion</span>
<span class="p">(</span><span class="nf">goto-char</span> <span class="nv">comint-last-input-end</span><span class="p">)</span>
<span class="p">(</span><span class="nv">comint-update-fence</span><span class="p">)))))))</span>
</pre>
<p>Echoes are canceled by adhering to the following procedure:</p>
<ul class="simple">
<li>Waiting for process output until enough characters have been emitted</li>
<li>Comparing the emitted text with the last user input</li>
<li>Only if they match that echoed text is deleted</li>
<li>A hack is applied to not delete the prompt</li>
</ul>
<p>Unfortunately my output filter is run before that, so it makes the
last check fail. I can only wonder whether it’s even possible to
use this API meaningfully and whether it will involve breaking
changes. Yet everyone and their dog keep proclaiming loudly how great
Emacs and its approach to text processing are…</p>
<table class="docutils footnote" frame="void" id="id3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id1">[1]</a></td><td><tt class="docutils literal">term.el</tt> is out because it doesn’t offer anything that
deserves to be called an API, <tt class="docutils literal">eshell.el</tt> doesn’t even have
documentation and is huge, <tt class="docutils literal">shell.el</tt> is small and simple.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="id4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id2">[2]</a></td><td>I recommend adding a <tt class="docutils literal"><span class="pre">(sit-for</span> 1)</tt> between functions doing
buffer manipulation to visualize what’s going on in the
buffer. Note that edebug supports doing this for everything
by switching to <tt class="docutils literal"><span class="pre">edebug-trace-mode</span></tt>.</td></tr>
</tbody>
</table>
tag:https://emacshorrors.com,2019-04-23:/posts/comint-process-echoes.html2019-04-23T22:09:46+02:00comint-process-echoes2019-04-23T22:09:46+02:00Vasilij Schneidermann
<p><strong>Update</strong>: <a class="reference external" href="http://lists.gnu.org/archive/html/bug-gnu-emacs/2018-06/msg00720.html">Bug report</a> thread with a workaround.</p>
<p><em>(This is a contributed post by</em> <a class="reference external" href="https://github.com/thblt/">thblt</a> <em>)</em></p>
<p>Trivia: How can you determine if the current Emacs instance has the
Emacs server running?</p>
<p>A quick search gives us three potential candidates: <tt class="docutils literal"><span class="pre">server-mode</span></tt>,
<tt class="docutils literal">(daemonp)</tt> and <tt class="docutils literal"><span class="pre">(server-running-p)</span></tt>. That’s way too much, but
surely one of them is the right one, isn’t it? Well, no. Because the
real answer to this trivial question is: <em>you can’t</em>.</p>
<ul class="simple">
<li><tt class="docutils literal"><span class="pre">server-mode</span></tt> is <tt class="docutils literal">t</tt> if, and only if, the server was started
using the function with the same name. But there are other ways to
run the server, like <tt class="docutils literal"><span class="pre">M-x</span> <span class="pre">server-start</span></tt> or <tt class="docutils literal">emacs <span class="pre">--daemon</span></tt>.</li>
<li><tt class="docutils literal">(daemonp)</tt> returns t if, and only if, Emacs was started in daemon
mode.</li>
</ul>
<p>What about <tt class="docutils literal"><span class="pre">(server-running-p)</span></tt>, then? Well, it may look friendly,
but here be monsters.</p>
<p>It starts by looking promising: after <tt class="docutils literal"><span class="pre">M-x</span> <span class="pre">server-start</span></tt>,
<tt class="docutils literal"><span class="pre">(server-running-p)</span></tt> now returns <tt class="docutils literal">t</tt>! Do we have a winner? Not yet!
Let’s pop a <em>new</em> Emacs instance and eval <tt class="docutils literal"><span class="pre">(server-running-p)</span></tt> without
starting the server. <tt class="docutils literal">t</tt> again!</p>
<p>What’s happening? The truth is that <tt class="docutils literal"><span class="pre">(server-running-p)</span></tt> is not
what it seems to be. Here’s its complete source code:</p>
<pre class="code elisp literal-block">
<span class="p">(</span><span class="nb">defun</span> <span class="nv">server-running-p</span> <span class="p">(</span><span class="kp">&optional</span> <span class="nv">name</span><span class="p">)</span>
<span class="s">"Test whether server NAME is running.
Return values:
nil the server is definitely not running.
t the server seems to be running.
something else we cannot determine whether it's running without using
commands which may have to wait for a long time."</span>
<span class="p">(</span><span class="nb">unless</span> <span class="nv">name</span> <span class="p">(</span><span class="k">setq</span> <span class="nv">name</span> <span class="nv">server-name</span><span class="p">))</span>
<span class="p">(</span><span class="k">condition-case</span> <span class="no">nil</span>
<span class="p">(</span><span class="k">if</span> <span class="nv">server-use-tcp</span>
<span class="p">(</span><span class="nb">with-temp-buffer</span>
<span class="p">(</span><span class="nv">insert-file-contents-literally</span> <span class="p">(</span><span class="nf">expand-file-name</span> <span class="nv">name</span> <span class="nv">server-auth-dir</span><span class="p">))</span>
<span class="p">(</span><span class="k">or</span> <span class="p">(</span><span class="k">and</span> <span class="p">(</span><span class="nf">looking-at</span> <span class="s">"127\\.0\\.0\\.1:[0-9]+ \\([0-9]+\\)"</span><span class="p">)</span>
<span class="p">(</span><span class="nf">assq</span> <span class="ss">'comm</span>
<span class="p">(</span><span class="nf">process-attributes</span>
<span class="p">(</span><span class="nf">string-to-number</span> <span class="p">(</span><span class="nv">match-string</span> <span class="mi">1</span><span class="p">))))</span>
<span class="no">t</span><span class="p">)</span>
<span class="nb">:other</span><span class="p">))</span>
<span class="p">(</span><span class="nf">delete-process</span>
<span class="p">(</span><span class="nf">make-network-process</span>
<span class="nb">:name</span> <span class="s">"server-client-test"</span> <span class="nb">:family</span> <span class="ss">'local</span> <span class="nb">:server</span> <span class="no">nil</span> <span class="nb">:noquery</span> <span class="no">t</span>
<span class="nb">:service</span> <span class="p">(</span><span class="nf">expand-file-name</span> <span class="nv">name</span> <span class="nv">server-socket-dir</span><span class="p">)))</span>
<span class="no">t</span><span class="p">)</span>
<span class="p">(</span><span class="nv">file-error</span> <span class="no">nil</span><span class="p">)))</span>
</pre>
<p>The horror starts as soon as the docstring. The <tt class="docutils literal"><span class="pre">-p</span></tt> suffix in the
name promises a predicate, that is, a boolean function. But in
<tt class="docutils literal"><span class="pre">server-running-p</span></tt>, non-<tt class="docutils literal">nil</tt> is not a loud and clear “Yes!”, it’s a
mumbled “well, maybe, who knows?”. Ternary logic, because Emacs is
above the law of excluded middle.</p>
<p>But what does this function <em>do</em>? It tries to determine if a server
called <tt class="docutils literal">NAME</tt> is running, by assuming that this server would be
configured exactly the same as the running instance. It may end up
looking at the socket file of the current server, or it may try to
initiate a TCP connection, which is extremely expensive.
<tt class="docutils literal"><span class="pre">server-running-p</span></tt> is the kind of function you may be tempted to
call while building the mode line: try it, and get an instant and
unrecoverable Emacs freeze. What it’s supposed to be useful for is
extremely unclear. It’s unable to determine if the running instance
has a server — but it uses this server’s config to search for a
potentially completely different server.</p>
tag:https://emacshorrors.com,2018-06-20:/posts/determining-if-the-server-is-started-or-the-wonders-of-server-running-p.html2018-06-20T09:51:21+02:00Determining if the server is started, or the wonders of server-running-p2018-06-20T09:51:21+02:00Vasilij Schneidermann
<p>It’s halloween, so here’s a real treat for you, the commentary in
<a class="reference external" href="http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/bytecomp.el">bytecomp.el</a>! The author of that piece of code is Jamie Zawinski who
did invaluable work for both GNU Emacs and XEmacs, these days he runs
a night club and <a class="reference external" href="https://www.jwz.org/blog/">blogs</a>. Here are my favorite parts of the file:</p>
<ul>
<li><pre class="code elisp first literal-block">
<span class="s">";; We successfully didn't compile this file."</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="p">(</span><span class="nf">insert</span> <span class="s">"\n"</span><span class="p">)</span> <span class="c1">; aaah, unix.</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="p">(</span><span class="nb">when</span> <span class="nv">old-style-backquotes</span>
<span class="p">(</span><span class="nv">byte-compile-warn</span> <span class="s">"!! The file uses old-style backquotes !!
This functionality has been obsolete for more than 10 years already
and will be removed soon. See (elisp)Backquote in the manual."</span><span class="p">))</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="c1">;; Insert semicolons as ballast, so that byte-compile-fix-header</span>
<span class="c1">;; can delete them so as to keep the buffer positions</span>
<span class="c1">;; constant for the actual compiled code.</span>
<span class="s">";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n"</span>
<span class="s">";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n"</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="c1">;; To avoid consing up monstrously large forms at load time, we split</span>
<span class="c1">;; the output regularly.</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="c1">;; If things not being bound at all is ok, so must them being</span>
<span class="c1">;; obsolete. Note that we add to the existing lists since Tramp</span>
<span class="c1">;; (ab)uses this feature.</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="c1">;; If foo.el declares `toto' as obsolete, it is likely that foo.el will</span>
<span class="c1">;; actually use `toto' in order for this obsolete variable to still work</span>
<span class="c1">;; correctly, so paradoxically, while byte-compiling foo.el, the presence</span>
<span class="c1">;; of a make-obsolete-variable call for `toto' is an indication that `toto'</span>
<span class="c1">;; should not trigger obsolete-warnings in foo.el.</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="c1">;; FIXME: we also use this hunk-handler to implement the function's dynamic</span>
<span class="c1">;; docstring feature. We could actually implement it more elegantly in</span>
<span class="c1">;; byte-compile-lambda so it applies to all lambdas, but the problem is that</span>
<span class="c1">;; the resulting .elc format will not be recognized by make-docfile, so</span>
<span class="c1">;; either we stop using DOC for the docstrings of preloaded elc files (at the</span>
<span class="c1">;; cost of around 24KB on 32bit hosts, double on 64bit hosts) or we need to</span>
<span class="c1">;; build DOC in a more clever way (e.g. handle anonymous elements).</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="c1">;; Don't reload the source version of the files below</span>
<span class="c1">;; because that causes subsequent byte-compilation to</span>
<span class="c1">;; be a lot slower and need a higher max-lisp-eval-depth,</span>
<span class="c1">;; so it can cause recompilation to fail.</span>
</pre>
</li>
<li><pre class="code elisp first literal-block">
<span class="c1">;; To avoid "lisp nesting exceeds max-lisp-eval-depth" when bytecomp compiles</span>
<span class="c1">;; itself, compile some of its most used recursive functions (at load time).</span>
</pre>
</li>
</ul>
<p>Don’t get me wrong, I’m aware that these are all necessary and don’t
indicate deeper faults in the source code. I merely find it
interesting what hacks one has to come up with for byte-code
compilation and found studying the file enlightening to say the least.</p>
tag:https://emacshorrors.com,2017-10-31:/posts/bytecompel.html2017-10-31T08:44:03+01:00bytecomp.el2017-10-31T08:44:03+01:00Vasilij Schneidermann
<p>I finally made <a class="reference external" href="https://github.com/wasamasa/nov.el">that EPUB mode</a>. This adventure mostly taught me
that eww, or rather, shr.el isn’t quite reusable. That itself is not
really a problem, but I <a class="reference external" href="https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28402">handed in a patch</a> to improve the situation.
An old saying among programmers is that every problem can be solved by
applying an extra level of indirection, so that’s what I did after
discussing it out on the bug tracker, however after my patch got
merged it was deemed <a class="reference external" href="http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/net/shr.el?id=658853aebb0ae2ee243276e04a7672fa7525ec5c#n505">too much</a>:</p>
<pre class="code elisp literal-block">
<span class="c1">;; We don't use shr-indirect-call here, since shr-descend is</span>
<span class="c1">;; the central bit of shr.el, and should be as fast as</span>
<span class="c1">;; possible. Having one more level of indirection with its</span>
<span class="c1">;; negative effect on performance is deemed unjustified in</span>
<span class="c1">;; this case.</span>
</pre>
<p>Hadn’t I spoken up about inclusion of this comment, an unsuspecting
future hacker wouldn’t even know why there’s duplicated code not using
the helper. I can only wonder how production-ready browser engines
solve this kind of problem…</p>
tag:https://emacshorrors.com,2017-10-19:/posts/unjustified-indirection.html2017-10-19T23:44:23+02:00Unjustified Indirection2017-10-19T23:44:23+02:00Vasilij Schneidermann
<p><strong>Update</strong>: Reddit points out that this has been fixed <a class="reference external" href="http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=a6ad98ad66e1d0c0dac5f25ba91e11d0cf9da725">on master</a> by
replacing most of the code with a call to gnulib’s <tt class="docutils literal">gen_tempname</tt>.</p>
<p>For someone not terribly experienced in writing safe programs, one can
only hope that building blocks like <tt class="docutils literal"><span class="pre">make-temp-file</span></tt> are doing the
right thing and cannot be subverted by a malicious third party. The
general advice here is that it’s preferable to use the primitive for
creating the temporary file instead of the primitive to generate its
name. Now, does Emacs reuse <tt class="docutils literal">mkstemp(3)</tt> for this? Or at least
<tt class="docutils literal">tmpnam(3)</tt>? Of course not! Where we go, we can just invent <a class="reference external" href="http://git.savannah.gnu.org/cgit/emacs.git/tree/src/fileio.c#n626">our
own source of randomness</a>:</p>
<p><tt class="docutils literal"><span class="pre">make-temp-file</span></tt> looks <a class="reference external" href="http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/files.el?id=0083123499cc29e301c197218d3809b225675e57#n1407">as follows</a>:</p>
<pre class="code c literal-block">
<span class="k">static</span> <span class="k">const</span> <span class="kt">char</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="mi">64</span><span class="p">]</span> <span class="o">=</span>
<span class="p">{</span>
<span class="sc">'A'</span><span class="p">,</span><span class="sc">'B'</span><span class="p">,</span><span class="sc">'C'</span><span class="p">,</span><span class="sc">'D'</span><span class="p">,</span><span class="sc">'E'</span><span class="p">,</span><span class="sc">'F'</span><span class="p">,</span><span class="sc">'G'</span><span class="p">,</span><span class="sc">'H'</span><span class="p">,</span>
<span class="sc">'I'</span><span class="p">,</span><span class="sc">'J'</span><span class="p">,</span><span class="sc">'K'</span><span class="p">,</span><span class="sc">'L'</span><span class="p">,</span><span class="sc">'M'</span><span class="p">,</span><span class="sc">'N'</span><span class="p">,</span><span class="sc">'O'</span><span class="p">,</span><span class="sc">'P'</span><span class="p">,</span>
<span class="sc">'Q'</span><span class="p">,</span><span class="sc">'R'</span><span class="p">,</span><span class="sc">'S'</span><span class="p">,</span><span class="sc">'T'</span><span class="p">,</span><span class="sc">'U'</span><span class="p">,</span><span class="sc">'V'</span><span class="p">,</span><span class="sc">'W'</span><span class="p">,</span><span class="sc">'X'</span><span class="p">,</span>
<span class="sc">'Y'</span><span class="p">,</span><span class="sc">'Z'</span><span class="p">,</span><span class="sc">'a'</span><span class="p">,</span><span class="sc">'b'</span><span class="p">,</span><span class="sc">'c'</span><span class="p">,</span><span class="sc">'d'</span><span class="p">,</span><span class="sc">'e'</span><span class="p">,</span><span class="sc">'f'</span><span class="p">,</span>
<span class="sc">'g'</span><span class="p">,</span><span class="sc">'h'</span><span class="p">,</span><span class="sc">'i'</span><span class="p">,</span><span class="sc">'j'</span><span class="p">,</span><span class="sc">'k'</span><span class="p">,</span><span class="sc">'l'</span><span class="p">,</span><span class="sc">'m'</span><span class="p">,</span><span class="sc">'n'</span><span class="p">,</span>
<span class="sc">'o'</span><span class="p">,</span><span class="sc">'p'</span><span class="p">,</span><span class="sc">'q'</span><span class="p">,</span><span class="sc">'r'</span><span class="p">,</span><span class="sc">'s'</span><span class="p">,</span><span class="sc">'t'</span><span class="p">,</span><span class="sc">'u'</span><span class="p">,</span><span class="sc">'v'</span><span class="p">,</span>
<span class="sc">'w'</span><span class="p">,</span><span class="sc">'x'</span><span class="p">,</span><span class="sc">'y'</span><span class="p">,</span><span class="sc">'z'</span><span class="p">,</span><span class="sc">'0'</span><span class="p">,</span><span class="sc">'1'</span><span class="p">,</span><span class="sc">'2'</span><span class="p">,</span><span class="sc">'3'</span><span class="p">,</span>
<span class="sc">'4'</span><span class="p">,</span><span class="sc">'5'</span><span class="p">,</span><span class="sc">'6'</span><span class="p">,</span><span class="sc">'7'</span><span class="p">,</span><span class="sc">'8'</span><span class="p">,</span><span class="sc">'9'</span><span class="p">,</span><span class="sc">'-'</span><span class="p">,</span><span class="sc">'_'</span>
<span class="p">};</span>
<span class="k">static</span> <span class="kt">unsigned</span> <span class="n">make_temp_name_count</span><span class="p">,</span> <span class="n">make_temp_name_count_initialized_p</span><span class="p">;</span>
<span class="cm">/* Value is a temporary file name starting with PREFIX, a string.
The Emacs process number forms part of the result, so there is
no danger of generating a name being used by another process.
In addition, this function makes an attempt to choose a name
which has no existing file. To make this work, PREFIX should be
an absolute file name.
BASE64_P means add the pid as 3 characters in base64
encoding. In this case, 6 characters will be added to PREFIX to
form the file name. Otherwise, if Emacs is running on a system
with long file names, add the pid as a decimal number.
This function signals an error if no unique file name could be
generated. */</span>
<span class="n">Lisp_Object</span>
<span class="nf">make_temp_name</span> <span class="p">(</span><span class="n">Lisp_Object</span> <span class="n">prefix</span><span class="p">,</span> <span class="kt">bool</span> <span class="n">base64_p</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">Lisp_Object</span> <span class="n">val</span><span class="p">,</span> <span class="n">encoded_prefix</span><span class="p">;</span>
<span class="kt">ptrdiff_t</span> <span class="n">len</span><span class="p">;</span>
<span class="n">printmax_t</span> <span class="n">pid</span><span class="p">;</span>
<span class="kt">char</span> <span class="o">*</span><span class="n">p</span><span class="p">,</span> <span class="o">*</span><span class="n">data</span><span class="p">;</span>
<span class="kt">char</span> <span class="n">pidbuf</span><span class="p">[</span><span class="n">INT_BUFSIZE_BOUND</span> <span class="p">(</span><span class="n">printmax_t</span><span class="p">)];</span>
<span class="kt">int</span> <span class="n">pidlen</span><span class="p">;</span>
<span class="n">CHECK_STRING</span> <span class="p">(</span><span class="n">prefix</span><span class="p">);</span>
<span class="cm">/* VAL is created by adding 6 characters to PREFIX. The first
three are the PID of this process, in base 64, and the second
three are incremented if the file already exists. This ensures
262144 unique file names per PID per PREFIX. */</span>
<span class="n">pid</span> <span class="o">=</span> <span class="n">getpid</span> <span class="p">();</span>
<span class="k">if</span> <span class="p">(</span><span class="n">base64_p</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">pidbuf</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">pid</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">pid</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="n">pidbuf</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">pid</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">pid</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="n">pidbuf</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">pid</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">pid</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="n">pidlen</span> <span class="o">=</span> <span class="mi">3</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">else</span>
<span class="p">{</span>
<span class="cp">#ifdef HAVE_LONG_FILE_NAMES
</span> <span class="n">pidlen</span> <span class="o">=</span> <span class="n">sprintf</span> <span class="p">(</span><span class="n">pidbuf</span><span class="p">,</span> <span class="s">"%"</span><span class="n">pMd</span><span class="p">,</span> <span class="n">pid</span><span class="p">);</span>
<span class="cp">#else
</span> <span class="n">pidbuf</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">pid</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">pid</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="n">pidbuf</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">pid</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">pid</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="n">pidbuf</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">pid</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">pid</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="n">pidlen</span> <span class="o">=</span> <span class="mi">3</span><span class="p">;</span>
<span class="cp">#endif
</span> <span class="p">}</span>
<span class="n">encoded_prefix</span> <span class="o">=</span> <span class="n">ENCODE_FILE</span> <span class="p">(</span><span class="n">prefix</span><span class="p">);</span>
<span class="n">len</span> <span class="o">=</span> <span class="n">SBYTES</span> <span class="p">(</span><span class="n">encoded_prefix</span><span class="p">);</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">make_uninit_string</span> <span class="p">(</span><span class="n">len</span> <span class="o">+</span> <span class="mi">3</span> <span class="o">+</span> <span class="n">pidlen</span><span class="p">);</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">SSDATA</span> <span class="p">(</span><span class="n">val</span><span class="p">);</span>
<span class="n">memcpy</span> <span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">SSDATA</span> <span class="p">(</span><span class="n">encoded_prefix</span><span class="p">),</span> <span class="n">len</span><span class="p">);</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">data</span> <span class="o">+</span> <span class="n">len</span><span class="p">;</span>
<span class="n">memcpy</span> <span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">pidbuf</span><span class="p">,</span> <span class="n">pidlen</span><span class="p">);</span>
<span class="n">p</span> <span class="o">+=</span> <span class="n">pidlen</span><span class="p">;</span>
<span class="cm">/* Here we try to minimize useless stat'ing when this function is
invoked many times successively with the same PREFIX. We achieve
this by initializing count to a random value, and incrementing it
afterwards.
We don't want make-temp-name to be called while dumping,
because then make_temp_name_count_initialized_p would get set
and then make_temp_name_count would not be set when Emacs starts. */</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">make_temp_name_count_initialized_p</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">make_temp_name_count</span> <span class="o">=</span> <span class="n">time</span> <span class="p">(</span><span class="nb">NULL</span><span class="p">);</span>
<span class="n">make_temp_name_count_initialized_p</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">while</span> <span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">unsigned</span> <span class="n">num</span> <span class="o">=</span> <span class="n">make_temp_name_count</span><span class="p">;</span>
<span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">num</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">num</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">num</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">num</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="n">p</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="n">make_temp_name_tbl</span><span class="p">[</span><span class="n">num</span> <span class="o">&</span> <span class="mi">63</span><span class="p">],</span> <span class="n">num</span> <span class="o">>>=</span> <span class="mi">6</span><span class="p">;</span>
<span class="cm">/* Poor man's congruential RN generator. Replace with
++make_temp_name_count for debugging. */</span>
<span class="n">make_temp_name_count</span> <span class="o">+=</span> <span class="mi">25229</span><span class="p">;</span>
<span class="n">make_temp_name_count</span> <span class="o">%=</span> <span class="mi">225307</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">check_existing</span> <span class="p">(</span><span class="n">data</span><span class="p">))</span>
<span class="p">{</span>
<span class="cm">/* We want to return only if errno is ENOENT. */</span>
<span class="k">if</span> <span class="p">(</span><span class="n">errno</span> <span class="o">==</span> <span class="n">ENOENT</span><span class="p">)</span>
<span class="k">return</span> <span class="n">DECODE_FILE</span> <span class="p">(</span><span class="n">val</span><span class="p">);</span>
<span class="k">else</span>
<span class="cm">/* The error here is dubious, but there is little else we
can do. The alternatives are to return nil, which is
as bad as (and in many cases worse than) throwing the
error, or to ignore the error, which will likely result
in looping through 225307 stat's, which is not only
dog-slow, but also useless since eventually nil would
have to be returned anyway. */</span>
<span class="n">report_file_error</span> <span class="p">(</span><span class="s">"Cannot create temporary name for prefix"</span><span class="p">,</span>
<span class="n">prefix</span><span class="p">);</span>
<span class="cm">/* not reached */</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">DEFUN</span> <span class="p">(</span><span class="s">"make-temp-name"</span><span class="p">,</span> <span class="n">Fmake_temp_name</span><span class="p">,</span> <span class="n">Smake_temp_name</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span>
<span class="nl">doc</span><span class="p">:</span> <span class="cm">/* Generate temporary file name (string) starting with PREFIX (a string).
The Emacs process number forms part of the result, so there is no
danger of generating a name being used by another Emacs process
\(so long as only a single host can access the containing directory...).
This function tries to choose a name that has no existing file.
For this to work, PREFIX should be an absolute file name.
There is a race condition between calling `make-temp-name' and creating the
file, which opens all kinds of security holes. For that reason, you should
normally use `make-temp-file' instead. */</span><span class="p">)</span>
<span class="p">(</span><span class="n">Lisp_Object</span> <span class="n">prefix</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="n">make_temp_name</span> <span class="p">(</span><span class="n">prefix</span><span class="p">,</span> <span class="mi">0</span><span class="p">);</span>
<span class="p">}</span>
</pre>
<p>The generated file name is therefore a combination of the prefix, the
Emacs PID and three characters from the above table. This makes about
200.000 possible temporary files that can be generated with a given
prefix in an Emacs session. This range can be traversed in a
negligible amount of time to recreate the state of the RNG and
accurately predict the next temporary file name.</p>
<pre class="code elisp literal-block">
<span class="p">(</span><span class="nb">defun</span> <span class="nv">make-temp-file</span> <span class="p">(</span><span class="nv">prefix</span> <span class="kp">&optional</span> <span class="nv">dir-flag</span> <span class="nv">suffix</span><span class="p">)</span>
<span class="s">"Create a temporary file.
The returned file name (created by appending some random characters at the end
of PREFIX, and expanding against </span><span class="ss">`temporary-file-directory'</span><span class="s"> if necessary),
is guaranteed to point to a newly created empty file.
You can then use </span><span class="ss">`write-region'</span><span class="s"> to write new data into the file.
If DIR-FLAG is non-nil, create a new empty directory instead of a file.
If SUFFIX is non-nil, add that at the end of the file name."</span>
<span class="c1">;; Create temp files with strict access rights. It's easy to</span>
<span class="c1">;; loosen them later, whereas it's impossible to close the</span>
<span class="c1">;; time-window of loose permissions otherwise.</span>
<span class="p">(</span><span class="nb">with-file-modes</span> <span class="sc">?\7</span><span class="mi">00</span>
<span class="p">(</span><span class="k">let</span> <span class="p">(</span><span class="nv">file</span><span class="p">)</span>
<span class="p">(</span><span class="k">while</span> <span class="p">(</span><span class="k">condition-case</span> <span class="p">()</span>
<span class="p">(</span><span class="k">progn</span>
<span class="p">(</span><span class="k">setq</span> <span class="nv">file</span>
<span class="p">(</span><span class="nf">make-temp-name</span>
<span class="p">(</span><span class="k">if</span> <span class="p">(</span><span class="nv">zerop</span> <span class="p">(</span><span class="nf">length</span> <span class="nv">prefix</span><span class="p">))</span>
<span class="p">(</span><span class="nf">file-name-as-directory</span>
<span class="nv">temporary-file-directory</span><span class="p">)</span>
<span class="p">(</span><span class="nf">expand-file-name</span> <span class="nv">prefix</span>
<span class="nv">temporary-file-directory</span><span class="p">))))</span>
<span class="p">(</span><span class="k">if</span> <span class="nv">suffix</span>
<span class="p">(</span><span class="k">setq</span> <span class="nv">file</span> <span class="p">(</span><span class="nf">concat</span> <span class="nv">file</span> <span class="nv">suffix</span><span class="p">)))</span>
<span class="p">(</span><span class="k">if</span> <span class="nv">dir-flag</span>
<span class="p">(</span><span class="nv">make-directory</span> <span class="nv">file</span><span class="p">)</span>
<span class="p">(</span><span class="nf">write-region</span> <span class="s">""</span> <span class="no">nil</span> <span class="nv">file</span> <span class="no">nil</span> <span class="ss">'silent</span> <span class="no">nil</span> <span class="ss">'excl</span><span class="p">))</span>
<span class="no">nil</span><span class="p">)</span>
<span class="p">(</span><span class="nv">file-already-exists</span> <span class="no">t</span><span class="p">))</span>
<span class="c1">;; the file was somehow created by someone else between</span>
<span class="c1">;; `make-temp-name' and `write-region', let's try again.</span>
<span class="no">nil</span><span class="p">)</span>
<span class="nv">file</span><span class="p">)))</span>
</pre>
<p>It’s interesting that the docstring of this function states that the
return value “is guaranteed to point to a newly created empty file.”.
If there were to exist a file for every possible combination for a
prefix, this function would just fall into an infinite loop and block
Emacs for no apparent reason. Both of these issues have been solved
in a better way in <a class="reference external" href="https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/posix/tempname.c;h=b00bd588ec458cbe3bc9bd162515995c0104248b;hb=HEAD">glibc</a>.</p>
<p>At least the impact of predicting the name is lessened if one uses
<tt class="docutils literal"><span class="pre">make-temp-file</span></tt> instead of <tt class="docutils literal"><span class="pre">make-temp-name</span></tt> on its own. An
attacker cannot create a symlink pointing to a rogue location with the
predicted name as that would trigger a <tt class="docutils literal"><span class="pre">file-already-exists</span></tt> error
and make the function use the next random name. All they could do is
read out the file afterwards iff they have the same permission as the
user Emacs runs with. A symlink attack can only be executed
successfully with a careless <tt class="docutils literal"><span class="pre">make-temp-name</span></tt> user, thankfully I’ve
not been able to find one worth subverting on GitHub yet.</p>
<p>Thanks to <tt class="docutils literal">dale</tt> on <tt class="docutils literal">#emacs</tt> for bringing this to my attention!</p>
tag:https://emacshorrors.com,2017-08-13:/posts/make-temp-name.html2017-08-13T20:37:32+02:00make-temp-name2017-08-13T20:37:32+02:00