Hosting Static Content with netlify.app

Many people associate netlify.com with its integration to GitHub, GitLab or Bitbucket. But you can just deploy your local files to Netlify as well.

Install netlify command via npm install netlify. cd to the directory where you locate your static content. Login to Netlify site via

netlify login

This will either directly open a browser, where you confirm the login, or enter your credentials. If the browser is not opened, then enter the URL given on console. Once you are logged in and have configured your domain name in the Netlify administration menu, you can deploy. For this run below command

netlify deploy --prod -d .

Netlify has the notion of two environments: preview and production. You can skip the preview environment and deploy directly to production.

The netlify command offers the following options.

$ netlify -h
Netlify command line tool

VERSION
  netlify-cli/2.48.0 linux-x64 node-v13.13.0

USAGE
  $ netlify [COMMAND]

COMMANDS
  addons     (Beta) Manage Netlify Add-ons
  api        Run any Netlify API method
  build      (Beta) Build on your local machine
  deploy     Create a new deploy from the contents of a folder
  dev        Local dev server
  functions  Manage netlify functions
  help       display help for netlify
  init       Configure continuous deployment for a new or existing site
  link       Link a local repo or project folder to an existing site on Netlify
  login      Login to your Netlify account
  open       Open settings for the site linked to the current folder
  plugins    list installed plugins
  sites      Handle various site operations
  status     Print status information
  switch     Switch your active Netlify account
  unlink     Unlink a local folder from a Netlify site
  watch      Watch for site deploy to finish

The netlify script stores your credentials in $HOME/.netlify/config.json.

Hosting Static Content with now.sh

now.sh, previously known under zeit.co, which has now rebranded as vercel.com, allows to host static content. There is no PHP, MySQL/MariaDB, Perl, CGI, etc. While surge.sh is super simple to use, in contrast now.sh uses the notion of ‘environment’, which can be either development, preview, or production.

First install via npm install now, then cd to the directory where you have stored your static content. Then, depending on your environment, deployment is as follows:

  1. Development: now dev, no “real” deployment, but rather web-server is started at localhost:3000. Stop with Ctrl-C.
  2. Preview: now or now deploy
  3. Production: now deploy --prod

It is not required to step through all the environments in any order. So you can just always deploy to production. Regardless of the environment, now.sh always creates some additional HTML web area with a generated name. This name might look like now-ll1keyjfk.now.sh.

The now command has the following options:

$ now -h                                                                                                           
> UPDATE AVAILABLE Run `npm i now@latest` to install Now CLI 18.0.0                                                                                
> Changelog: https://github.com/zeit/now/releases/tag/now@18.0.0                                                                                   
                                                                                                                                                   
  đťš« now [options] <command | path>                                                                                                                 
                                                                                                                                                   
  Commands:                                                                                                                                        
                                                                                                                                                   
    Basic                                                                                                                                          
                                                                                                                                                   
      deploy               [path]      Performs a deployment (default)                                                                             
      dev                              Start a local development server                                                                            
      init                 [example]   Initialize an example project                                                                               
      ls | list            [app]       Lists deployments                                                                                           
      inspect              [id]        Displays information related to a deployment                                                                
      login                [email]     Logs into your account or creates a new one                                                                 
      logout                           Logs out of your account                                                                                    
      switch               [scope]     Switches between teams and your personal account                                                            
      help                 [cmd]       Displays complete help for [cmd]                                                                            
                                                                                                                                                   
    Advanced                                                                                                                                       
                                                                                                                                                   
      rm | remove          [id]        Removes a deployment                                                                                        
      domains              [name]      Manages your domain names                                                                                   
      dns                  [name]      Manages your DNS records                                                                                    
      certs                [cmd]       Manages your SSL certificates                                                                               
      secrets              [name]      Manages your secret environment variables                                                                   
      logs                 [url]       Displays the logs for a deployment                                                                          
      teams                            Manages your teams                                                                                          
      whoami                           Shows the username of the currently logged in user                                                          
                                                                                                                                                   
  Options:

    -h, --help                     Output usage information
    -v, --version                  Output the version number
    -V, --platform-version         Set the platform version to deploy to
    -n, --name                     Set the project name of the deployment 
    -A FILE, --local-config=FILE   Path to the local `now.json` file
    -Q DIR, --global-config=DIR    Path to the global `.now` directory
    -d, --debug                    Debug mode [off]
    -f, --force                    Force a new deployment even if nothing has changed
    -t TOKEN, --token=TOKEN        Login token
    -p, --public                   Deployment is public (`/_src` is exposed)
    -e, --env                      Include an env var during run time (e.g.: `-e KEY=value`). Can appear many times.
    -b, --build-env                Similar to `--env` but for build time only.
    -m, --meta                     Add metadata for the deployment (e.g.: `-m KEY=value`). Can appear many times.
    -C, --no-clipboard             Do not attempt to copy URL to clipboard
    -S, --scope                    Set a custom scope
    --regions                      Set default regions to enable the deployment on
    --prod                         Create a production deployment

  > NOTE: To view the usage information for Now 1.0, run `now help deploy-v1`

  Examples:

  – Deploy the current directory

    $ now

  – Deploy a custom path

    $ now /usr/src/project

  – Deploy with environment variables

    $ now -e NODE_ENV=production -e SECRET=@mysql-secret

  – Show the usage information for the sub command `list`

    $ now help list

now.sh stores your credentials in $HOME/.local/share/now/auth.json.

Hosting Static Content with surge.sh

When you want totally hassle free hosting of static HTML then surge.sh is very attractive. It is easy to set-up and free of charge for most private users. It offers https out of the box from sectigo.com. It does not offer PHP, MySQL/MariaDB, CGI, Perl, etc. Just static HTML with CSS, JavaScript, images, etc. Your static content will be hosted on Your_Domain.surge.sh.

Steps to follow:

  1. Install surge: npm install surge
  2. cd to your directory with static content: Type surge

It cannot be easier. If you do not want to enter the domain name over and over again, you can store this chosen domain name in file CNAME and you won’t be asked the next time:

echo Your_Domain > CNAME

The surge command offers the following options.

$ surge --help

  surge – single command web publishing. (v0.21.3)

  Usage:
    surge <project> <domain>

  Options:
    -a, --add           adds user to list of collaborators (email address)
    -r, --remove        removes user from list of collaborators (email address)
    -V, --version       show the version number
    -h, --help          show this help message

  Additional commands:
    surge whoami        show who you are logged in as
    surge logout        expire local token
    surge login         only performs authentication step
    surge list          list all domains you have access to
    surge teardown      tear down a published project
    surge plan          set account plan

  Guides:
    Getting started     surge.sh/help/getting-started-with-surge
    Custom domains      surge.sh/help/adding-a-custom-domain
    Additional help     surge.sh/help

  When in doubt, run surge from within your project directory.

Your e-mail and encrypted password are stored in $HOME/.netrc.

Performance Comparison Pallene vs. Lua 5.1, 5.2, 5.3, 5.4 vs. C

Installing Pallene is described in the previous post: Installing Pallene Compiler. In this post we test the performance of Pallene versus C, Lua 5.4, and LuaJIT.

1. Array Access. I checked a similar program as in Performance Comparison C vs. Lua vs. LuaJIT vs. Java.

function lua_perf(N:integer, S:integer)
        local t:{ {a:float, b:float, f:float} } = {}

        for i = 1, N do
                t[i] = {
                        a = 0.0,
                        b = 1.0,
                        f = i * 0.25
                }
        end

        for j = 1, S-1 do
                for i = 1, N-1 do
                        t[i].a = t[i].a + t[i].b * t[i].f
                        t[i].b = t[i].b - t[i].a * t[i].f
                end
                --io_write( t[1].a )
        end
end

This program, which does no I/O at all, runs in 0.14s, and therefore runs two times slower than the LuaJIT, which finishes in 0.07s. This clearly is somewhat disappointing. Lua 5.4, as part of Pallene, needs 0.75s. So Pallene is roughly five times faster than Lua.
Continue reading

Installing Pallene Compiler

Pallene is a Lua based language. In contrast to Lua, which is untyped, Pallene is typed. A good paper on Pallene is “Pallene: A companion language for Lua”, by Hugo Musso Gualandi, and Roberto Ierusalimschy.

From above paper:

The compiler itself is quite conventional. After a standard parsing step, it converts the program to a high-level intermediate form and from that it emits C code, which is then fed into a C compiler such as gcc.

From “A gradually typed subset of a scripting language can be simple and efficient”:

Pallene was designed for performance, and one fundamental part of that is that its compiler generates efficient machine code. To simplify the implementation, and for portability, Pallene generates C source code instead of directly generating assembly language.

So, very generally, this idea is similar to f2c (Fortran to C), cobc (Cobol compiler), or Lush (Lisp Universal SHell).

The whole Pallene compiler is implemented in less than 7 kLines of Lua, and less than 1 kLines of C source code for the runtime.

To install Pallene compiler you need git, gcc, lua, and luarocks. Description is for Linux. MacOS is very similar.

1. Source. Fetch source code via git clone.

$ git clone https://github.com/pallene-lang/pallene.git 
Cloning into 'pallene'...
$ cd pallene

2. Rocks. Fetch required Lua rocks via luarocks command.

$ luarocks install --local --only-deps pallene-dev-1.rockspec
Missing dependencies for pallene dev-1:                                                                                                                 
   lpeglabel >= 1.5.0 (not installed)                                                                                                                   
   inspect >= 3.1.0 (not installed)                                                                                                                     
   argparse >= 0.7.0 (not installed)                                                                                                                    
   luafilesystem >= 1.7.0 (not installed)                                                                                                               
   chronos >= 0.2 (not installed)                                                                                                                       
                                                                                                                                                        
pallene dev-1 depends on lua ~> 5.3 (5.3-1 provided by VM)                                                                                              
pallene dev-1 depends on lpeglabel >= 1.5.0 (not installed)                                                                                             
Installing https://luarocks.org/lpeglabel-1.6.0-1.src.rock                                                                                              
                                                                                                                                                        
lpeglabel 1.6.0-1 depends on lua >= 5.1 (5.3-1 provided by VM)
gcc -O2 -fPIC -I/usr/include -c lpcap.c -o lpcap.o
gcc -O2 -fPIC -I/usr/include -c lpcode.c -o lpcode.o
gcc -O2 -fPIC -I/usr/include -c lpprint.c -o lpprint.o
gcc -O2 -fPIC -I/usr/include -c lptree.c -o lptree.o
gcc -O2 -fPIC -I/usr/include -c lpvm.c -o lpvm.o
gcc -shared -o lpeglabel.so lpcap.o lpcode.o lpprint.o lptree.o lpvm.o
No existing manifest. Attempting to rebuild...
lpeglabel 1.6.0-1 is now installed in /home/klm/.luarocks (license: MIT/X11) 

pallene dev-1 depends on inspect >= 3.1.0 (not installed)
Installing https://luarocks.org/inspect-3.1.1-0.src.rock

inspect 3.1.1-0 depends on lua >= 5.1 (5.3-1 provided by VM)
inspect 3.1.1-0 is now installed in /home/klm/.luarocks (license: MIT <http://opensource.org/licenses/MIT>)

pallene dev-1 depends on argparse >= 0.7.0 (not installed)
Installing https://luarocks.org/argparse-0.7.0-1.all.rock

argparse 0.7.0-1 depends on lua >= 5.1, < 5.4 (5.3-1 provided by VM)
argparse 0.7.0-1 is now installed in /home/klm/.luarocks (license: MIT)

pallene dev-1 depends on luafilesystem >= 1.7.0 (not installed)
Installing https://luarocks.org/luafilesystem-1.8.0-1.src.rock

luafilesystem 1.8.0-1 depends on lua >= 5.1 (5.3-1 provided by VM)
gcc -O2 -fPIC -I/usr/include -c src/lfs.c -o src/lfs.o
gcc -shared -o lfs.so src/lfs.o
luafilesystem 1.8.0-1 is now installed in /home/klm/.luarocks (license: MIT/X11)

pallene dev-1 depends on chronos >= 0.2 (not installed)
Installing https://luarocks.org/chronos-0.2-4.src.rock

chronos 0.2-4 depends on lua >= 5.1 (5.3-1 provided by VM)
gcc -O2 -fPIC -I/usr/include -c src/chronos.c -o src/chronos.o -I/usr/include
gcc -shared -o chronos.so src/chronos.o -L/usr/lib -Wl,-rpath,/usr/lib -lrt
chronos 0.2-4 is now installed in /home/klm/.luarocks (license: MIT/X11)

Stopping after installing dependencies for pallene dev-1

3. Environment variables. Make sure that you source the environment variables given by

luarocks path

For example:

export LUA_PATH='/usr/share/lua/5.3/?.lua;/usr/share/lua/5.3/?/init.lua;/usr/lib/lua/5.3/?.lua;/usr/lib/lua/5.3/?/init.lua;./?.lua;./?/init.lua;/home/klm/.luarocks/share/lua/5.3/?.lua;/home/klm/.luarocks/share/lua/5.3/?/init.lua'
export LUA_CPATH='/usr/lib/lua/5.3/?.so;/usr/lib/lua/5.3/loadall.so;./?.so;/home/klm/.luarocks/lib/lua/5.3/?.so'
export PATH='/home/klm/.luarocks/bin:/usr/bin:/home/klm/bin:...:.

4. Build Lua and runtime. Build Lua and the Pallene runtime (you are still in the pallene directory):

make linux-readline

Some warnings will show up for Lua, but they can be ignored for now.

5. Run compiler. Now you can run pallenec, provided you still are in the same directory, where you built pallene.

$ ./pallenec   
Usage: pallenec [-h] [--emit-c] [--emit-asm] [--compile-c]
       [--dump {parser,checker,ir,uninitialized,constant_propagation}]
       <source_file>

Error: missing argument 'source_file'

6. Run example. Now check one of the examples.

$ pallenec examples/factorial/factorial.pln
$ ./lua/src/lua -l factorial examples/factorial/main.lua 
The factorial of 5 is 120.

The most common error will be to not use the lua/src/lua command from Pallene, but rather the system-wide.

You can compile all examples and benchmarks:

for i in examples/*/*.pln; do pallenec $i; done
for i in benchmark/*/*.pln; do pallenec $i; done

Things to note in Pallene:

  1. Array indexes must start at one
  2. Pallene source code, except type-, record-definition or variable definitions, must be within a function
  3. Pallene offers no goto statement. The goto statement was added in Lua 5.2.

Performance Comparison in Computing Exponential Function

If your computation is dominated by exponential function evaluations, then it makes a significant difference whether you evaluate the exponential function exp() in single precision or in double precision. You can reduce your computing time by roughly 25% when moving from double precision (double) to single precision (float). Evaluation in quadruple precision is more than six times more expensive than evaluation in double precision.

Changing from double precision to single precision also halves the amount of storage needed. On x86_64 Linux float usually occupies 4 bytes, double occupies 8 bytes, and long double needs 16 bytes.

1. Result. Here are the runtime numbers of a test program.

  1. Single precision (float): 2.44s
  2. Double precision (double): 3.32s
  3. Quadruple precision (long double): 22.88s

These numbers are dependant on CPU internal scheduling, see CPU Usage Time Is Dependant on Load.

2. Test program. The test program is essentially as below:

long i, rep=1024, n=65000;
int c, precision='d';
float sf = 0;
double sd = 0;
long double sq = 0;
...
switch(precision) {
case 'd':
        while (rep-- > 0)
                for (i=0; i<n; ++i)
                        sd += exp(i % 53) - exp((i+1) % 43) - exp((i+2) % 47) - exp((i+3) % 37);
        printf("sd = %f\n",sd);
        break;
case 'f':
        while (rep-- > 0)
                for (i=0; i<n; ++i)
                        sf += expf(i % 53) - expf((i+1) % 43) - expf((i+2) % 47) - expf((i+3) % 37);
        printf("sf = %f\n",sf);
        break;
case 'q':
        while (rep-- > 0)
                for (i=0; i<n; ++i)
                        sq += expl(i % 53) - expl((i+1) % 43) - expl((i+2) % 47) - expl((i+3) % 37);
        printf("sq = %Lf\n",sq);
        break;
}

Full source code is in GitHub, file in question is called exptst.c.

3. Environment.AMD Bulldozer FX-8120, 3.1 GHz, Arch Linux 5.6.8, gcc version 9.3.0. Compiled the code with -O3 -march=native

youtube-dl HTTP 403 error

youtube-dl is a set of Python scripts for downloading videos from YouTube, Twitter, Vimeo, and many other sites, see for example the list of supported sites. This is a handy tool if you want to watch videos or listen to music on your smartphone: Download them with youtube-dl and store them on your smartphone. Also see Youtube video to mp3.

I had some issues downloading a YouTube video using youtube-dl.

$ youtube-dl https://youtu.be/26QTzeOV8Gs 
[ youtube ]  26QTzeOV8Gs: Downloading webpage
ERROR: unable to download video data: HTTP Error 403: Forbidden

The following command did the trick:

$ youtube-dl --rm-cache-dir
Removing cache dir /home/klm/.cache/youtube-dl ...