deactivate-mark

18/06/2016

This is a codeless post that will instead focus on a design issue present in all (at the time of writing) stable releases of Emacs. Be assured that you will not have to work around it in the upcoming Emacs 25 release.

Have you ever wondered why some commands deactivate the region afterwards, although there’s no explicit call to the deactivate-mark function? It turns out that this is intentional behavior as can be seen in the documentation of the deactivate-mark variable:

If an editing command sets this to t, deactivate the mark afterward.
The command loop sets this to nil before each command,
and tests the value when the command returns.
Buffer modification stores t in this variable.

So, any command modifying a buffer will deactivate the region. Makes sense and if you for some reason need the region again, it’s a C-x C-x away. There is a major problem with this though, it doesn’t matter which buffer is modified…

This bit me hard with eyebrowse. I am using a modeline indicator to visualize its state which is using the built-in format-spec package. As that package is using a temporary buffer for turning a format string into a formatted string and the modeline indicator is recalculated very often, this led to the region being deactivated on any command. It took me quite a bit to figure this one out. I consider it madness for anyone to expect this behavior when writing functions that should not interfere with the region, so I’m glad it has been fixed in Emacs 25 by making the variable buffer-local.


Let's consider an obsolete replacement

08/06/2016

I’m currently writing my second mode, this time for textual markup. As I still don’t have much experience with it, I did look at other modes of that kind, ultimately ending up with rst.el.

It’s not unusual for older code to redefine things that could possibly not supported by all Emacs versions out there. What I did not expect however, was an implementation of symbolic regular expressions:

(defvar rst-re-alist) ; Forward declare to use it in `rst-re'.

;; FIXME: Use `sregex' or `rx' instead of re-inventing the wheel.
(rst-testcover-add-compose 'rst-re)
;; testcover: ok.
(defun rst-re (&rest args)
  "Interpret ARGS as regular expressions and return a regex string.
Each element of ARGS may be one of the following:

A string which is inserted unchanged.

A character which is resolved to a quoted regex.

A symbol which is resolved to a string using `rst-re-alist-def'.

A list with a keyword in the car.  Each element of the cdr of such
a list is recursively interpreted as ARGS.  The results of this
interpretation are concatenated according to the keyword.

For the keyword `:seq' the results are simply concatenated.

For the keyword `:shy' the results are concatenated and
surrounded by a shy-group (\"\\(?:...\\)\").

For the keyword `:alt' the results form an alternative (\"\\|\")
which is shy-grouped (\"\\(?:...\\)\").

For the keyword `:grp' the results are concatenated and form a
referenceable group (\"\\(...\\)\").

After interpretation of ARGS the results are concatenated as for
`:seq'."
  (apply 'concat
         (mapcar
          (lambda (re)
            (cond
             ((stringp re)
              re)
             ((symbolp re)
              (cadr (assoc re rst-re-alist)))
             ((characterp re)
              (regexp-quote (char-to-string re)))
             ((listp re)
              (let ((nested
                     (mapcar (lambda (elt)
                               (rst-re elt))
                             (cdr re))))
                (cond
                 ((eq (car re) :seq)
                  (mapconcat 'identity nested ""))
                 ((eq (car re) :shy)
                  (concat "\\(?:" (mapconcat 'identity nested "") "\\)"))
                 ((eq (car re) :grp)
                  (concat "\\(" (mapconcat 'identity nested "") "\\)"))
                 ((eq (car re) :alt)
                  (concat "\\(?:" (mapconcat 'identity nested "\\|") "\\)"))
                 (t
                  (error "Unknown list car: %s" (car re))))))
             (t
              (error "Unknown object type for building regex: %s" re))))
          args)))

;; FIXME: Remove circular dependency between `rst-re' and `rst-re-alist'.
(with-no-warnings ; Silence byte-compiler about this construction.
  (defconst rst-re-alist
    ;; Shadow global value we are just defining so we can construct it step by
    ;; step.
    (let (rst-re-alist)
      (dolist (re rst-re-alist-def rst-re-alist)
        (setq rst-re-alist
              (nconc rst-re-alist
                     (list (list (car re) (apply 'rst-re (cdr re))))))))
    "Alist mapping symbols from `rst-re-alist-def' to regex strings."))

I find it hilarious that they appear to be aware of a now obsolete alternative and a more powerful, officially supported one, yet decided to do their own thang. At least there’s not much code around that could be yucky, if you ignore that one circular dependency mentioned at the bottom between the function and its look-up alist.


"One XNOOP in 100 loops will make Emacs terminate"

05/06/2016
/* On some systems, an X bug causes Emacs to get no more events
   when the window is destroyed.  Detect that.  (1994.)  */
if (! event_found)
  {
    /* Emacs and the X Server eats up CPU time if XNoOp is done every time.
       One XNOOP in 100 loops will make Emacs terminate.
       B. Bretthauer, 1994 */
    x_noop_count++;
    if (x_noop_count >= 100)
      {
        x_noop_count=0;

        if (next_noop_dpyinfo == 0)
          next_noop_dpyinfo = x_display_list;

        XNoOp (next_noop_dpyinfo->display);

        /* Each time we get here, cycle through the displays now open.  */
        next_noop_dpyinfo = next_noop_dpyinfo->next;
      }
  }

I had some difficulties believing that an over 20 years old comment still holds true, so I snipped this functionality out for fun:

--- a/xterm.c       2016-06-05 11:38:14.471347194 +0200
+++ b/xterm.c       2016-06-05 11:36:37.398817889 +0200
@@ -8786,26 +8786,10 @@ XTread_socket (struct terminal *terminal
     }
 #endif /* USE_GTK */

-  /* On some systems, an X bug causes Emacs to get no more events
-     when the window is destroyed.  Detect that.  (1994.)  */
   if (! event_found)
     {
-      /* Emacs and the X Server eats up CPU time if XNoOp is done every time.
-    One XNOOP in 100 loops will make Emacs terminate.
-    B. Bretthauer, 1994 */
       x_noop_count++;
-      if (x_noop_count >= 100)
-   {
-     x_noop_count=0;
-
-     if (next_noop_dpyinfo == 0)
-       next_noop_dpyinfo = x_display_list;
-
-     XNoOp (next_noop_dpyinfo->display);
-
-     /* Each time we get here, cycle through the displays now open.  */
-     next_noop_dpyinfo = next_noop_dpyinfo->next;
-   }
+      fprintf(stderr, "XNoOp %d occurred\n", x_noop_count);
     }

   /* If the focus was just given to an auto-raising frame,

No difference when used on a GTK or Lucid build. Other than nearly every event causing the diagnostic message to print…


Judge, Condemn and Redeem

26/05/2016
/* The following three hooks are used when we're doing a thorough
   redisplay of the frame.  We don't explicitly know which scroll bars
   are going to be deleted, because keeping track of when windows go
   away is a real pain - "Can you say set-window-configuration, boys
   and girls?"  Instead, we just assert at the beginning of redisplay
   that *all* scroll bars are to be removed, and then save a scroll bar
   from the fiery pit when we actually redisplay its window.  */

/* Arrange for all scroll bars on FRAME to be removed at the next call
   to `*judge_scroll_bars_hook'.  A scroll bar may be spared if
   `*redeem_scroll_bar_hook' is applied to its window before the judgment.  */

static void
w32_condemn_scroll_bars (struct frame *frame)
{
  if (!NILP (FRAME_SCROLL_BARS (frame)))
    {
      if (!NILP (FRAME_CONDEMNED_SCROLL_BARS (frame)))
        {
          /* Prepend scrollbars to already condemned ones.  */
          Lisp_Object last = FRAME_SCROLL_BARS (frame);

          while (!NILP (XSCROLL_BAR (last)->next))
            last = XSCROLL_BAR (last)->next;

          XSCROLL_BAR (last)->next = FRAME_CONDEMNED_SCROLL_BARS (frame);
          XSCROLL_BAR (FRAME_CONDEMNED_SCROLL_BARS (frame))->prev = last;
        }

      fset_condemned_scroll_bars (frame, FRAME_SCROLL_BARS (frame));
      fset_scroll_bars (frame, Qnil);
    }
}


/* Un-mark WINDOW's scroll bar for deletion in this judgment cycle.
   Note that WINDOW isn't necessarily condemned at all.  */

static void
w32_redeem_scroll_bar (struct window *w)
{
  struct scroll_bar *bar;
  Lisp_Object barobj;
  struct frame *f;

  /* We can't redeem this window's scroll bar if it doesn't have one.  */
  if (NILP (w->vertical_scroll_bar) && NILP (w->horizontal_scroll_bar))
    emacs_abort ();

  if (!NILP (w->vertical_scroll_bar) && WINDOW_HAS_VERTICAL_SCROLL_BAR (w))
    {
      bar = XSCROLL_BAR (w->vertical_scroll_bar);
      /* Unlink it from the condemned list.  */
      f = XFRAME (WINDOW_FRAME (w));
      if (NILP (bar->prev))
        {
          /* If the prev pointer is nil, it must be the first in one of
             the lists.  */
          if (EQ (FRAME_SCROLL_BARS (f), w->vertical_scroll_bar))
            /* It's not condemned.  Everything's fine.  */
            goto horizontal;
          else if (EQ (FRAME_CONDEMNED_SCROLL_BARS (f),
                       w->vertical_scroll_bar))
            fset_condemned_scroll_bars (f, bar->next);
          else
            /* If its prev pointer is nil, it must be at the front of
               one or the other!  */
            emacs_abort ();
        }
      else
        XSCROLL_BAR (bar->prev)->next = bar->next;

      if (! NILP (bar->next))
        XSCROLL_BAR (bar->next)->prev = bar->prev;

      bar->next = FRAME_SCROLL_BARS (f);
      bar->prev = Qnil;
      XSETVECTOR (barobj, bar);
      fset_scroll_bars (f, barobj);
      if (! NILP (bar->next))
        XSETVECTOR (XSCROLL_BAR (bar->next)->prev, bar);
    }

 horizontal:
  if (!NILP (w->horizontal_scroll_bar) && WINDOW_HAS_HORIZONTAL_SCROLL_BAR (w))
    {
      bar = XSCROLL_BAR (w->horizontal_scroll_bar);
      /* Unlink it from the condemned list.  */
      f = XFRAME (WINDOW_FRAME (w));
      if (NILP (bar->prev))
        {
          /* If the prev pointer is nil, it must be the first in one of
             the lists.  */
          if (EQ (FRAME_SCROLL_BARS (f), w->horizontal_scroll_bar))
            /* It's not condemned.  Everything's fine.  */
            return;
          else if (EQ (FRAME_CONDEMNED_SCROLL_BARS (f),
                       w->horizontal_scroll_bar))
            fset_condemned_scroll_bars (f, bar->next);
          else
            /* If its prev pointer is nil, it must be at the front of
               one or the other!  */
            emacs_abort ();
        }
      else
        XSCROLL_BAR (bar->prev)->next = bar->next;

      if (! NILP (bar->next))
        XSCROLL_BAR (bar->next)->prev = bar->prev;

      bar->next = FRAME_SCROLL_BARS (f);
      bar->prev = Qnil;
      XSETVECTOR (barobj, bar);
      fset_scroll_bars (f, barobj);
      if (! NILP (bar->next))
        XSETVECTOR (XSCROLL_BAR (bar->next)->prev, bar);
    }
}

/* Remove all scroll bars on FRAME that haven't been saved since the
   last call to `*condemn_scroll_bars_hook'.  */

static void
w32_judge_scroll_bars (struct frame *f)
{
  Lisp_Object bar, next;

  bar = FRAME_CONDEMNED_SCROLL_BARS (f);

  /* Clear out the condemned list now so we won't try to process any
     more events on the hapless scroll bars.  */
  fset_condemned_scroll_bars (f, Qnil);

  for (; ! NILP (bar); bar = next)
    {
      struct scroll_bar *b = XSCROLL_BAR (bar);

      x_scroll_bar_remove (b);

      next = b->next;
      b->next = b->prev = Qnil;
    }

  /* Now there should be no references to the condemned scroll bars,
     and they should get garbage-collected.  */
}

Source.


C-C-COMPILE IT IN

02/05/2016

I am implementing MAL (again), this time in ChucK. Why an audio programming language? Well, as I wielded it to implement a pseudo-theremin, it dawned upon me that it exposed more than enough language features to allow for implementing something lispier in it. A good chance to learn OOP, I thought to myself[1].

It’s not too fun to write loads of code in a language without a proper major mode for it[2], so I decided to give CC Mode a try, this time for creating a mode with it. Half a day and loads of cursing later, I was done. Most of my gripes with it stemmed from the documentation not covering that use case (and the examples being outdated), so in case you plan to go down that road, I found the sources of d-mode and csharp-mode highly useful. I’ll concentrate on a single, innocent looking warning I got while developing it to show a less obvious reason for hating this piece of Emacs with the force of a thousand suns.

If you byte-compile chuck-mode.el (or FWIW, csharp-mode.el), you’ll get two warnings:

chuck-mode.el:152:4:Warning: (lambda nil ...) quoted with ' rather than with #'
chuck-mode.el:152:4:Warning: (lambda nil ...) quoted with ' rather than with #'

Following the line numbers isn’t particularly enlightening as they point to the wrong place. I was already familiar with these warnings as I’ve seen them before for csharp-mode, but this time I had something much smaller to bisect. Commenting out (c-init-language-vars) made them disappear, so I looked up its definition and as it’s a macro expanding to a trivial looking function call, that one’s source as well. Here it is, in all of its glory:

(defun c-make-init-lang-vars-fun (mode)
  "Create a function that initializes all the language dependent variables
for the given mode.

This function should be evaluated at compile time, so that the
function it returns is byte compiled with all the evaluated results
from the language constants.  Use the `c-init-language-vars' macro to
accomplish that conveniently."

  (if (cc-bytecomp-is-compiling)
      ;; No need to byte compile this lambda since the byte compiler is
      ;; smart enough to detect the `funcall' construct in the
      ;; `c-init-language-vars' macro below and compile it all straight
      ;; into the function that contains `c-init-language-vars'.
      `(lambda ()

         ;; This let sets up the context for `c-mode-var' and similar
         ;; that could be in the result from `c--macroexpand-all'.
         (let ((c-buffer-is-cc-mode ',mode)
               current-var source-eval)
           (c-make-emacs-variables-local)
           (condition-case err

               (if (eq c-version-sym ',c-version-sym)
                   (setq ,@(let ((c-buffer-is-cc-mode mode)
                                 (c-lang-const-expansion 'immediate))
                             ;; `c-lang-const' will expand to the evaluated
                             ;; constant immediately in `c--macroexpand-all'
                             ;; below.
                              (c--mapcan
                               (lambda (init)
                                 `(current-var ',(car init)
                                               ,(car init) ,(c--macroexpand-all
                                                             (elt init 1))))
                               ;; Note: The following `append' copies the
                               ;; first argument.  That list is small, so
                               ;; this doesn't matter too much.
                               (append (cdr c-emacs-variable-inits)
                                       (cdr c-lang-variable-inits)))))

                 ;; This diagnostic message isn't useful for end
                 ;; users, so it's disabled.
                 ;;(unless (get ',mode 'c-has-warned-lang-consts)
                 ;;  (message ,(concat "%s compiled with CC Mode %s "
                 ;;                    "but loaded with %s - evaluating "
                 ;;                    "language constants from source")
                 ;;           ',mode ,c-version c-version)
                 ;;  (put ',mode 'c-has-warned-lang-consts t))

                 (setq source-eval t)
                 (let ((init ',(append (cdr c-emacs-variable-inits)
                                       (cdr c-lang-variable-inits))))
                   (dolist (var-init init)
                     (setq current-var (car var-init))
                     (set (car var-init) (eval (cadr var-init))))))

             (error
              (if current-var
                  (message "Eval error in the `c-lang-defvar' or `c-lang-setvar' for `%s'%s: %S"
                           current-var
                           (if source-eval
                               (format "\
 (fallback source eval - %s compiled with CC Mode %s but loaded with %s)"
                                       ',mode ,c-version c-version)
                             "")
                           err)
                (signal (car err) (cdr err)))))))

    ;; Being evaluated from source.  Always use the dynamic method to
    ;; work well when `c-lang-defvar's in this file are reevaluated
    ;; interactively.
    `(lambda ()
       (require 'cc-langs)
       (let ((c-buffer-is-cc-mode ',mode)
             (init (append (cdr c-emacs-variable-inits)
                           (cdr c-lang-variable-inits)))
             current-var)
         (c-make-emacs-variables-local)
         (condition-case err

             (dolist (var-init init)
               (setq current-var (car var-init))
               (set (car var-init) (eval (cadr var-init))))

           (error
            (if current-var
                (message
                 "Eval error in the `c-lang-defvar' or `c-lang-setver' for `%s' (source eval): %S"
                 current-var err)
              (signal (car err) (cdr err)))))))
    ))

For clarification, CC Mode expects you to define constants for your language. The above monstrosity turns these constants into values applied to each buffer using the derived mode, but does it differently, depending on whether you are in the process of byte-compiling its file or just load it (like, for re-evaluation while doing some development). As the mechanism for that backquotes a lambda and replaces parts of it, the byte-compiler will naturally warn us about it as it prohibits it from byte-compiling the lambda (unlike what the comment suggests).

This means that any differences between the two implementations will have fun side effects, such as you no longer being able to test your mode meaningfully by re-evaluating parts of it (forcing you to recompile and load, ideally in a new instance). When it comes to its worst, you’re going to have a broken mode just because it wasn’t compiled with a CC Mode compiled with the same Emacs version. Seriously. Don’t do that. It just angers people for no real reason. Do what every other major mode does and just set the damn variables buffer-locally. Thank you for your understanding.

[1]I found out quickly that while ChucK’s idea of OOP is clearly Java-inspired, it doesn’t implement nearly as much and is significantly more limited, so I ended up working against it most of the time.
[2]I got by initially with an abandoned mode, but found it annoying that it wasn’t defined properly (which made Evil start in the wrong state for me), had way too magic behaviour for = and terrible indentation. That’s why I picked this yak to shave…