How to do HTML/Javascript REPL (Read–eval–print loop) with NO server set up

Problem

I love the REPL (Read–eval–print loop). Watch live update of web page when typing code is cool.

But most solutions around the internet are not practical for me.

In my world, I never has a chance to develop fancy 2D/3D games written in HTML5 canvas.

Instead, I was ALWAYS assigned to maintain some enterprise CMS with complex business logic. For example, after refreshing the web page I need click lots of buttons before doing the job. Avoiding press "refresh page button" does not help much.

Besides, I can't use the embedded server those solutions boasted because our web application has many dependencies. It needs a whole environment support team to set it up. There is no way I can persuade the team to use a different web server

For me, a practical solution must be:

  • easy to set up
  • powerful. I can read/write any part of the web page in my text editor
  • no learning curve. I don't want to learn any domain specific language or API. For example, if my web application uses jquery, I want use and only use jquery API to manipulate web page in my text editor. If it use backbone, then I use backbone. No more, no less.

Solution

The best solution is firefox addon MozRepl. You can install it https://addons.mozilla.org. Click firefox menu "Tools => MozRepl => Start" to run it. moz-repl-start-nq8.png

So what is MozRepl, as its web site says, "MozRepl lets you program Firefox and other Mozilla-based applications from the inside." Basically it convert firefox into a socket server. Your text editor could talk to firefox and let firefox do anything you want. Since it only interact with the firefox, I don't need mess with the web server.

Well, I hope I can end this article by claiming "after reading its official tutorial,I live happily with MozRepl ever after".

The ugly truth is, MozRepl has some rough edges. That's the reason why I write this article. I will explain how to use MozRepl in next two sections.

Section 1 is about how to use MozRepl, whatever text editor you use, you should read this section.

Section 2 is on how to set up my text editor Emacs to interact with MozRepl. It's optional.

Section 1, use MozRepl

Here are key points:

  • You need read above official tutorial at least once
  • There is video demo on youtube. I suggest watching from 2:18 because you possibly are more interested in manipulating web page than toy with firefox internals.
  • web page is called "content" in MozRepl
  • In order to execute your javascript code with exactly same syntax as your web application is running code "repl.enter(content);" at first line, as you can see in both video and text tutorial
  • Use firefox 24.0+ and MozRepl 1.1.2+
  • For some weird reason, MozRepl socket server needs some warm up after web page refreshed. So run the code like "console.log('hello');" in the first connection, close the connection, and run your other code in second connection.
  • For code running in MozRepl, if one line will trigger some AJAX request, all the code just after that line will not be executed. Use javascript API "setTimeout' to work around this issue.

Here is a complete example (note how I use setTimeout) written in javascript,

$('#serviceId').val('0451235550');
$('#serviceId').keyup();
console.log('hi');
setTimeout(function(){
  console.log('world');
  $('input[value="1"][name="isAccountHolder"]').click();
  $('#accountNumber').val('01234567891234');
  setTimeout(function(){
    $('#registrationLoginEmailAddress').val('cb@cb.cb');
    $('#registrationLoginEmailAddress2').val('cb@cb.cb');
    $('#password').val('Password1');
    $('#passwordConfirm').val('Password1');
    $('#title').val('Mr');
    $('#serviceFirstName').val('chen');
    $('#serviceLastName').val('bin');
    $('#registrationLoginDetailsContinueBtn').click();
  },2000);
  $('#serviceAccountContinueBtn').click();
},3000);
$('#confirmServiceBtn').click();

As you can see from above, I can use jQuery API loaded in that web page without any problem. Of course, you can always use vanilla javascript instead.

Please note the code is not part of my web application! It will be run only in MozRepl. You can telnet to the MozRepl socket server (127.0.0.1 with port 4424) and copy paste above code line by line.

Section 2, set up text editor (Emacs)

My text editor is Emacs, so I use Emacs as an example.

Step 1, install moz.el according to official manual.

Step 2, Please insert below code into your ~/.emacs:

(defun moz-goto-content-and-run-cmd (cmd)
  (comint-send-string (inferior-moz-process)
                      (concat "repl.enter(content);"
                              cmd
                              "repl.back();")))

(setq moz-repl-js-dir (expand-file-name "~/moz-repl-js-dir"))

(defun moz--read-file (js-file)
  (with-temp-buffer
    (insert-file-contents js-file)
    (buffer-string)))

(defun moz--load-js-file (js-file)
  (let (cmd )
    (when (file-exists-p js-file)
      ;; make moz API usable in any major-mode
      (moz-minor-mode 1)
      ;; flush mozrepl at first
      (moz-goto-content-and-run-cmd "console.log('hello');")
      ;; read the content of js-file
      (setq cmd (moz--read-file js-file))
      (moz-goto-content-and-run-cmd cmd))))

(defun moz-load-js-file-and-send-it ()
  "load js file from specific directory and send it to mozrepl"
  (interactive)
  (let ((js-file (read-file-name "js file:" moz-repl-js-dir)))
    (moz--load-js-file js-file)
    ))

From now on, you only need "M-x moz-load-js-file-and-send-it" and select the js file you need MozRepl to run.

Tips

Tip 1

Well, this is a simple tip. Maybe you are already aware of this.

You can write a all-in-one.js. In all-in-one.js, you can parse the URL to separate logic for different project.

Here is javascript code,

var url=document.URL;
if(/^.*mycompany\.com\.au.*my-account\/registration/.test(url)){
   console.log("for regitration module");
} else if (/^.*mycompany\.com\.au.*my-account\/my-profile/.test(url)) {
   console.log("for profile module");
}

Since javascript is good at analyzing URL, you can also tweak the URL parameters. Anyway, the full power of javascript is at your hand.

Tip 2

Sometimes mozrepl will be unresponsive. You can test it by running "M-x moz-reload-browser" to refresh the page. If page is not refreshed, restart Firefox. Luckily this does not happen too often.

(defun moz-reload-browser ()
  (interactive)
  (comint-send-string (inferior-moz-process)
                      "setTimeout(function(){content.document.location.reload(true);}, '500');"))

Demo

Here is a demo how I use it in a real world web application development.

The script to drive the firefox is written in simple javascript/jquery code, which I already show you.

mozrepl-in-action.gif

Notes on set up nfs on ArchLinux

Key points


/srv/nfs4/z 192.168.0.0/16(ro,insecure,no_subtree_check)

# turn on the log

sudo rpcdebug -m nfsd -s proc

# turn off the log
  • then `sudo tail -f /var/log/messages.log`
  • Basically you need `sudo systemctl enable syslog-ng.service` to enable system log before you use rpcdebug

Summary

On server side:


sudo exportfs

# got output like:

# /srv/nfs4/z   192.168.0.0/16

Then on client side:


# 131072 is 128K

sudo mount 192.168.1.9:/srv/nfs4/z ~/z -o rsize=131072,wsize=131072,noatime,nodiratime,intr

Git merge in command line

List and filter all the files need resolve conflict, then apply certain git operation on them:


git diff --name-only --diff-filter=U|grep "\.html\|\.min\.js"|xargs -I{} sh -c "git checkout --theirs {} && git add {}"

Switch input method in evil-mode (在Evil中快速切换输入法)

I need input Chinese (or any other language) in Emacs. The default key binding is C-\ to toggle input method.

Since I'm using evil-mode (Vim simulation), I need switch evil state from normal to insert before switching input method.

The extra keybindings could be optimized by below code in ~/.emacs:

(defun evil-toggle-input-method ()
  "when toggle on input method, switch to evil-insert-state if possible.
when toggle off input method, switch to evil-normal-state if current state is evil-insert-state"
  (interactive)
  (if (not current-input-method)
      (if (not (string= evil-state "insert"))
          (evil-insert-state))
    (if (string= evil-state "insert")
        (evil-normal-state)))
  (toggle-input-method))

(global-set-key (kbd "C-\\") 'evil-toggle-input-method)

BTW, I recommend the Chinese input method pyim written in pure elisp.

Emacs中切换其自带输入法的快捷键是=C-\=,对应的命令是=M-x toggle-input-method=.

我遇到的问题是如果使用Evil(一种Vim模拟)则切换输入法不方便.我需要先切换evil-state至insert,然后才能切换输入法.

只要将以上代码贴入=~/.emacs=,则切换输入法可以更加高效:

顺便提一下,pyim是强烈推荐的中文输入法.

Effective spell check in Emacs

CREATED: <2014-01-21 Tue>

UPDATED: <2016-12-07 Wed>

I use Flyspell in web-mode as sample. Other major modes also works.

At first, please turn on flyspell for all programming languages by inserting below code into .emacs,

(add-hook 'prog-mode-hook 'flyspell-prog-mode)

There is also a short tutorial on:

  • How to setup flyspell
  • Difference between hunspell and aspell
  • How to setup hunspell

Spell check in HTML

I will spell check,

  • Text between html tag like <label>Please input email</label>
  • Value of html input control like <input type"text" value="Please input your name">=
  • CSS class name like <div class"btn btn-default" />=

My setup:

;; {{ flyspell setup for web-mode
(defun web-mode-flyspell-verify ()
  (let* ((f (get-text-property (- (point) 1) 'face))
         rlt)
    (cond
     ;; Check the words with these font faces, possibly.
     ;; this *blacklist* will be tweaked in next condition
     ((not (memq f '(web-mode-html-attr-value-face
                     web-mode-html-tag-face
                     web-mode-html-attr-name-face
                     web-mode-constant-face
                     web-mode-doctype-face
                     web-mode-keyword-face
                     web-mode-comment-face ;; focus on get html label right
                     web-mode-function-name-face
                     web-mode-variable-name-face
                     web-mode-css-property-name-face
                     web-mode-css-selector-face
                     web-mode-css-color-face
                     web-mode-type-face
                     web-mode-block-control-face)))
      (setq rlt t))
     ;; check attribute value under certain conditions
     ((memq f '(web-mode-html-attr-value-face))
      (save-excursion
        (search-backward-regexp "=['\"]" (line-beginning-position) t)
        (backward-char)
        (setq rlt (string-match "^\\(value\\|class\\|ng[A-Za-z0-9-]*\\)$"
                                (thing-at-point 'symbol)))))
     ;; finalize the blacklist
     (t
      (setq rlt nil)))
    rlt))
(put 'web-mode 'flyspell-mode-predicate 'web-mode-flyspell-verify)
;; }}

I use web-mode for HTML files. The technique applies on other modes (php-mode, html-mode ….) .

Don't display doublon (double word) as error

Modern CSS frameworks like Bootstrap make doublon unavoidable. For example, CSS class name btn btn-default contains double word btn.

So we need stop displaying doublon as error in HTML,

(defvar flyspell-check-doublon t
  "Check doublon (double word) when calling `flyspell-highlight-incorrect-region'.")
 (make-variable-buffer-local 'flyspell-check-doublon)

(defadvice flyspell-highlight-incorrect-region (around flyspell-highlight-incorrect-region-hack activate)
  (if (or flyspell-check-doublon (not (eq 'doublon (ad-get-arg 2))))
      ad-do-it))

(defun web-mode-hook-setup ()
  (flyspell-mode 1)
  (setq flyspell-check-doublon nil))

(add-hook 'web-mode-hook 'web-mode-hook-setup)

Spell check camel case strings

We can check camel cased string/variable/function if and only if aspell is used.

Insert below code into .emacs,

;; if (aspell installed) { use aspell}
;; else if (hunspell installed) { use hunspell }
;; whatever spell checker I use, I always use English dictionary
;; I prefer use aspell because:
;; 1. aspell is older
;; 2. looks Kevin Atkinson still get some road map for aspell:
;; @see http://lists.gnu.org/archive/html/aspell-announce/2011-09/msg00000.html
(setq ispell-program-name "aspell"
      ;; force the English dictionary, support Camel Case spelling check (tested with aspell 0.6)
      ispell-extra-args '("--sug-mode=ultra" "--lang=en_US" "--run-together")

Summary

EmacsWiki suggests (flyspell-prog-mode) which only checks typo in comments.

But as I proved, Emacs gives you full freedom to design a different solution.

Screen shot (typos are underscored):

spell-check-html-in-emacs.png

Javascript and ReactJS setup (OPTIONAL)

If you fully understand my previous sections, you don't need read this section.

Insert below code into .emacs,

(defun js-flyspell-verify ()
  (let* ((f (get-text-property (- (point) 1) 'face)))
    ;; *whitelist*
    ;; only words with following font face will be checked
    (memq f '(js2-function-call
              js2-function-param
              js2-object-property
              font-lock-variable-name-face
              font-lock-string-face
              font-lock-function-name-face))))
(put 'js2-mode 'flyspell-mode-predicate 'js-flyspell-verify)
(put 'rjsx-mode 'flyspell-mode-predicate 'js-flyspell-verify)

How to execute user program for System V init script

Memo for my self.


sudo ls -1 /etc/{rc,init}.d/|grep local

# then read the file either rc.local or local

Notes on Google Chrome's dev tool

command in source panel hotkey
Continue C-\
Toggle break point C-B
Step over C-'
Step into C-;
Step out S-C-;
Up call stack C-,
Down call stack C-.
Toggle console view ESC
Goto line C-G
Goto file C-P
Jump to method S-C-O
  • I don't use hotkey F1~F12 because my windows manager is different from most developers
  • live editing does not work on Chromium 30.0.1599.114 Built on Ubuntu 13.04. Anyway, I don't need this functionality
  • hotkeys of "Continue" button in Sources panel does not work on Chromium 30.0.1599.114 Built on Ubuntu 13.04.
  • "Break on" inspected element is extremely useful

My grub2 configuration

Content of /etc/grub.d/11_windows,

#!/bin/sh -e
echo "#        Adding Windows GRUB 2 menu"
cat << EOF
menuentry "Windows XP" {
savedefault
insmod chain
insmod ntfs
set root=(hd0,1)
chainloader +1
}
EOF

/etc/default/grub

# Copyright 1999-2013 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/sys-boot/grub/files/grub.default-2,v 1.4 2013/09/21 18:10:55 floppym Exp $
#
# To populate all changes in this file you need to regenerate your
# grub configuration file afterwards:
#     'grub2-mkconfig -o /boot/grub/grub.cfg'
#
# See the grub info page for documentation on possible variables and
# their associated values. 

GRUB_DISTRIBUTOR="Gentoo"

GRUB_DEFAULT=saved
GRUB_SAVEDEFAULT=true
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10

# Append parameters to the linux kernel command line
GRUB_CMDLINE_LINUX="libata.dma=1"

# Append parameters to the linux kernel command line for non-recovery entries
#GRUB_CMDLINE_LINUX_DEFAULT=""

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal.
# Note that you can use only modes which your graphic card supports via VBE.
# You can see them in real GRUB with the command `vbeinfo'.
#GRUB_GFXMODE=640x480

# Path to theme spec txt file.
# The starfield is by default provided with use truetype.
# NOTE: when enabling custom theme, ensure you have required font/etc.
#GRUB_THEME="/boot/grub/themes/starfield/theme.txt"

# Background image used on graphical terminal.
# Can be in various bitmap formats.
#GRUB_BACKGROUND="/boot/grub/mybackground.png"

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to kernel
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY=true

Why Emacs is better editor - a case study for javascript developer

UPDATED: <2014-01-15 Wed>

Let's see an example in real life development.

Note

For people who does not get the key points of this article, here are the points:

  • Emacs has an embedded javascript interpreter which I extend a little bit.
  • Other editors just use external tools or regular expression to do the semantic analysis.
  • The difference of above two is bigger than the difference between machine gun and spear.

Problem

I'm maintaining a big javascript file with thousands of lines of legacy code.

My most urgent issue is how to list all the functions in that javascript file and jump to the definition of specific function easily.

The modern enterprise application usually define the javascript function in some complex data structure. So most editors are not good at listing the functions.

To understand what "complex" means, let's check some code from real world application:

$(el.completeRegistrationForm).validate({
    ignore: " :hidden",
    rules : {
        password : {
            required : function () { return $(el.password).is(":visible"); }
        },
        accountNumber : {
            required : function () {
                return $(el.accountNumber).is(":visible");
            },
            digits : true
        }
        // ... I skipped next 200 lines which are similar to above lines

    },
    messages : {
        password: {
            required : "Please input a valid password"
        },
        accountNumber: {
            required : "Please provide a valid account number",
            digits : "Please enter only digits",
        }
        // ... I skipped next 200 lines which are similar to above lines
    }
});

Most editors like Sublime Text 3 cannot display the javascript functions with meaningful context in this case. All you can see is only a bunch of functions with name "required".: sublime-functions.png

Solution

Emacs has a js2-mode which is basically a javascript interpreter written in lisp. It's created by Steve Yegge and now maintained by mooz.

Since js2-mode is a interpreter, basically it can do anything you want. The key point is to understand the Abstract Syntax Tree (AST) defined in js2-mode.

Here is my patch to make js2-mode display the list of functions with correct context:

commit 56ed89bf18a6b58fd4620056288ea2ab52bd4d77
Author: Chen Bin <chenbin.sh@gmail.com>
Date:   Sun Dec 15 18:18:06 2013 +1100

    more hint for orphan function

diff --git a/js2-imenu-extras.el b/js2-imenu-extras.el
index e8e15a5..17bf158 100644
--- a/js2-imenu-extras.el
+++ b/js2-imenu-extras.el
@@ -174,6 +174,39 @@ prefix any functions defined inside the IIFE with the module name."
          (js2-imenu-record-module-pattern node)))
        t))))

+(defun js2-imenu-get-parent-keyname-list (node)
+  "get the list of keys of parent of node
+for example, for javascript code, {rules:{ password {required: function(){}}}}
+the return will be '(rules password)."
+  (let ((rlt '())
+        (n node))
+    (while (setq n (js2-imenu-parent-prop-node n))
+      (add-to-list 'rlt (js2-prop-node-name (js2-object-prop-node-left n)))
+      )
+    rlt
+    )
+  )
+
+(defun js2-imenu-parent-prop-node (node)
+  "for javascript code: parent-key-name:{ required:function(){} }
+we need know the  parent-key-name.
+ step1, 'required:function(){}' is the js2-object-prop-node
+ step2, '{ required:function(){} }' is the js2-object-node
+ step3, 'parent-key-name:{ required:function(){} }' is js2-object-prop-node
+"
+  (let (p2 p3)
+    ;; step 2
+    (setq p2 (js2-node-parent node))
+    ;; step 3
+    (when (and p2 (js2-object-node-p p2))
+      (setq p3 (js2-node-parent p2))
+      (if (and p3 (js2-object-prop-node-p p3))
+        p3
+        )
+      )
+    )
+  )
+
 (defun js2-imenu-record-orphan-function (node)
   "Record orphan function when it's the value of NODE.
 NODE must be `js2-object-prop-node'."
@@ -181,10 +214,15 @@ NODE must be `js2-object-prop-node'."
     (let ((fn-node (js2-object-prop-node-right node)))
       (unless (and js2-imenu-function-map
                    (gethash fn-node js2-imenu-function-map))
-        (let ((key-node (js2-object-prop-node-left node)))
-          (js2-record-imenu-entry fn-node
-                                  (list js2-imenu-other-functions-ns
-                                        (js2-prop-node-name key-node))
+        (let ((key-node (js2-object-prop-node-left node))
+              (parent-prop-node (js2-imenu-parent-prop-node node))
+              mylist
+              )
+          (setq mylist (append (js2-imenu-get-parent-keyname-list node)
+                               (list (js2-prop-node-name key-node))
+                               ))
+          (add-to-list 'mylist js2-imenu-other-functions-ns)
+          (js2-record-imenu-entry fn-node mylist
                                   (js2-node-abs-pos key-node)))))))

 (defun js2-imenu-record-module-pattern (node)

I already submitted the patch to mooz so everyone will enjoy this feature in the future.

Here is the screen shot of emacs in old js2-mode, the UI is based on Imenu Mode and Helm: emacs-functions.png

The screen shot after we applying the above patch: emacs-functions-improved.png

Summary

That's an example of beauty of Emacs.

It gives you freedom and power. You can base your work on the top geeks like Steve Yegge and mooz. Just a few lines of lisp code to kick ass.

Update

My patch is incorporated into js2-mode since version 20140114.

After installing js2-mode, you need paste one line setup into your ~/.emacs,

(js2-imenu-extras-mode)

The UI to display the candidate is from package Helm.

The Helm version should be 20140125.1101 or higher, you can install Helm from MELPA.

After installing Helm, you can use command M-x helm-imenu to show the list of functions to jump to. Here is the screen shot how I use imenu in my hello2.js:

helm-imenu-and-js2-mode-nq8.png

UPDATE: I suggest using counsel-imenu from Counsel instead of Helm.

Why some systemd service on ArchLinux does not work

List failed services

sudo systemctl --failed

I can also use `sudo journalctl -b` to get the log message when booting all the services. But not very useful.

Find the root cause of failed service

sudo systemctl status service-name

Why my /usr/lib/systemd/system/network-wifi.service does not work

Here is the content of my wifi service

[Unit]
Description=Wireless networkd connectivity (wlan0)
Wants=network.target
Before=network.target
BindsTo=sys-subsystem-net-devices-wlan0.device
After=sys-subsystem-net-devices-wlan0.device

[Service]
Type=oneshot
RemainAfterExit=yes
#EnvironmentFile=/etc/conf.d/network-wireless@wlan0

ExecStart=/usr/bin/ip link set wlan0 up
ExecStart=/usr/bin/wpa_supplicant -B -i wlan0 -c /etc/wpa_supplicant/yang.conf
ExecStart=/usr/bin/dhcpcd wlan0

ExecStop=/usr/bin/ip addr flush dev wlan0
ExecStop=/usr/bin/ip link set dev wlan0 down

[Install]
WantedBy=multi-user.target

It's because netcfg.service blocks my network-wifi.service.

Solutin is simple:

sudo systemctl disable netcfg

How to start a service as specific user (mpd, for example)

This is the content of /usr/lib/systemd/system/mpd.service:

[Unit]
Description=Music Player Daemon
After=network.target sound.target

[Service]
ExecStart=/usr/bin/mpd --no-daemon /home/cb/.mpd/config
User=cb

[Install]
WantedBy=multi-user.target