java.sql.SQLRecoverableException: IO Error: Connection reset by peer, Authentication lapse

I encountered the following error, when I wanted to connect to Oracle v12.2.0.1.0 database with Java 1.8.0_192-b26:

java.sql.SQLRecoverableException: IO Error: Connection reset by peer, Authentication lapse 321631 ms.

This was unexpected as the same program did run absolutely fine on another Linux machine. Program in question is

import java.sql.Connection;
import java.sql.SQLException;

import oracle.jdbc.pool.OracleDataSource;

public class OraSample1 {

        public static void main (String argv[]) {
                System.out.println("Starting...");
                try {
                        OracleDataSource ds = new OracleDataSource();
                        ds.setURL("jdbc:oracle:thin:@nuc:1521:orcl");
                        Connection conn=ds.getConnection("c##klm","klmOpRisk");
                        System.out.println("Connected");
                } catch (SQLException e) {
                        System.out.println(e);
                }
        }

}

Solution: Add the following property setting to command line

java -Djava.security.egd=file:/dev/urandom OraSample1

Also see “java.sql.SQLException: I/O Error: Connection reset” in linux server [duplicate] on Stackoverflow,

Passing HashMap from Java to Java Nashorn

Java Nashorn is the JavaScript engine shipped since Java 8. You can therefore use JavaScript wherever you have at least Java 8. Java 8 also has a standalone interpreter, called jjs.

It is possible to create a Java HashMap and use this structure directly in JavaScript. Here is the code:

import java.util.*;
import java.io.*;
import javax.script.*;


public class HashMapDemo {

        public static void main(String[] args) {
                HashMap hm = new HashMap();

                hm.put("A", new Double(3434.34));
                hm.put("B", new Double(123.22));
                hm.put("C", new Double(1200.34));
                hm.put("D", new Double(99.34));
                hm.put("E", new Double(-19.34));

                for( String name: hm.keySet() )
                        System.out.println(name + ": "+ hm.get(name));

                // Increase A's balance by 1000
                double balance = ((Double)hm.get("A")).doubleValue();
                hm.put("A", new Double(balance + 1000));
                System.out.println("A's new account balance : " + hm.get("A"));

                // Call JavaScript from Java
                try {   
                        ScriptEngine engine = new ScriptEngineManager().getEngineByName("nashorn");
                        engine.eval("print('Hello World');");
                        engine.eval(new FileReader("example.js"));
                        Invocable invocable = (Invocable) engine;
                        Object result = invocable.invokeFunction("sayHello", "John Doe");
                        System.out.println(result);
                        System.out.println(result.getClass());

                        result = invocable.invokeFunction("prtHash", hm);
                        System.out.println(result);
                } catch (FileNotFoundException | NoSuchMethodException | ScriptException e) {
                        e.printStackTrace();
                        System.out.println(e);
                }

        }
}

And here is the corresponding JavaScript file example.js:

var sayHello = function(name) {
        print('Hello, ' + name + '!');
        return 'hello from javascript';
};

var prtHash = function(h) {
        print('h.A = ' + h.A);
        print('h.B = ' + h["B"]);
        print('h.C = ' + h.C);
        print('h.D = ' + h["D"]);
        print('h.E = ' + h.E);
};

Output is:

$ java HashMapDemo
A: 3434.34
B: 123.22
C: 1200.34
D: 99.34
E: -19.34
A's new account balance : 4434.34
Hello World
Hello, John Doe!
hello from javascript
class java.lang.String
h.A = 4434.34
h.B = 123.22
h.C = 1200.34
h.D = 99.34
h.E = -19.34
null

Above example uses sample code from

  1. Riding the Nashorn: Programming JavaScript on the JVM
  2. Simple example for Java HashMap
  3. Nashorn: Run JavaScript on the JVM

Decisive was the statement in https://winterbe.com/posts/2014/04/05/java8-nashorn-tutorial/:

Java objects can be passed without loosing any type information on the javascript side. Since the script runs natively on the JVM we can utilize the full power of the Java API or external libraries on nashorn.

Above program works the same if one changes HashMap to HashMap and populating accordingly, e.g.:

                HashMap hm = new HashMap();

                hm.put("A", new Double(3434.34));
                hm.put("B", new String("Test"));
                hm.put("C", new Date(5000));
                hm.put("D", new Integer(99));
                hm.put("E", new Boolean(Boolean.TRUE));

Output from JavaScript would be

h.A = 4434.34
h.B = Test
h.C = Thu Jan 01 01:00:05 CET 1970
h.D = 99
h.E = true

Entries changed in JavaScript can be returned back to Java. Assume JavaScript program changes values:

var prtHash = function(h,hret) {
        hret.U = 57;
        hret.V = "Some text";
        hret.W = false;
};

Then these changed arguments can be used back in Java program:

HashMap hret = new HashMap();

result = invocable.invokeFunction("prtHash", hm, hret);
System.out.println(result);
System.out.println("hret.U = " + hret.get("U"));
System.out.println("hret.V = " + hret.get("V"));
System.out.println("hret.W = " + hret.get("W"));

Output is then

hret.U = 57
hret.V = Some text
hret.W = false

Using Scooter Software Beyond Compare

Beyond Compare is a graphical file comparison tool sold by Scooter Software. Its open-source competitors are mainly vimdiff, and kdiff3. Its advantage is ease-of-use. While comparing files they can be edited instantly. You can diff complete directory trees.

It is written in Delphi Object Pascal, the source code is not open-source. It runs on Windows, x86 Linux, and OS X. It does not run on ARM, like Raspberry Pi or Odroid, see support for arm processors – like the raspberry pi. The “Standard Edition” costs $30, the “Pro Edition” costs $60. The software is in AUR.

1. Root User Problem. When using it as root-user you must use:

export QT_GRAPHICSSYSTEM=native
bcompare

When running

DIFFPROG=bcompare pacdiff

the screen looks like this:

2. Git Usage. To use Beyond Compare with git difftool you have to do two things: First you must create an alias bc3 for bcompare.

[root /bin]# ln -s bcompare bc3

Second add the following lines to your ~/.gitconfig file:

[diff]
        tool = bc3
[difftool]
        prompt = false
        bc3 = trustExitCode
[merge]
        tool = bc3
[mergetool]
        bc3 = trustExitCode

Alternatively to above changes in the ~/.gitconfig file, use the following commands:

git config --global diff.tool bc3
git config --global difftool.bc3.trustExitCode true
git config --global merge.tool bc3
git config --global mergetool.bc3.trustExitCode true

Towards web-based delta synchronization for cloud storage systems

Very interesting article.

Some remarkable excerpts:

To isolate performance issues to the JavaScript VM, the authors rebuilt the client side of WebRsync using the Chrome native client support and C++. It’s much faster.

Replacing MD5 with SipHash reduces computation complexity by almost 5x. As a fail-safe mechanism in case of hash collisions, WebRsync+ also uses a lightweight full content hash check. If this check fails then the sync will be re-started using MD5 chunk fingerprinting instead.

The client side of WebR2sync+ is 1700 lines of JavaScript. The server side is based on node.js (about 500 loc) and a set of C processing modules (a further 1000 loc).

the morning paper

Towards web-based delta synchronization for cloud storage systems Xiao et al., FAST’18

If you use Dropbox (or an equivalent service) to synchronise file between your Mac or PC and the cloud, then it uses an efficient delta-sync (rsync) protocol to only upload the parts of a file that have changed. If you use a web interface to synchronise the same files though, the entire file will be uploaded. This situation seems to hold across a wide range of popular services:

Given the universal presence of the web browser, why can’t we have efficient delta syncing for web clients? That’s the question Xiao et al. set out to investigate: they built an rsync implementation for the web, and found out it performed terribly. Having tried everything to improve the performance within the original rsync design parameters, then they resorted to a redesign which moved more of the heavy lifting back to…

View original post 728 more words

Unix Command comm: Compare Two Files

One lesser known Unix command is comm. This command is far less known than diff. comm needs two already sorted files FILE1 and FILE2. With the options

  • -1 suppress column 1 (lines unique to FILE1)
  • -2 suppress column 2 (lines unique to FILE2)
  • -3 suppress column 3 (lines that appear in both files)

For example, comm -12 F1 F2 prints all common lines in files F1 and F2.

I thought that comm had a bug, so I wrote a short Perl script to simulate the behaviour of comm. Of course, there was no bug, I just missed to notice that the records in the two files did not match due to white space.

#!/bin/perl -W
use strict;

use Getopt::Std;
my %opts = ('d' => 0, 's' => 0);
getopts('ds:',\%opts);
my $debug = ($opts{'d'} != 0);
my $member = defined($opts{'s'}) ? $opts{'s'} : 0;

my ($set,$prev) = (1,"");
my %H;

while (<>) {
        $prev = $ARGV if ($prev eq "");
        if ($ARGV ne $prev) {
                $set *= 2;
                $prev = $ARGV;
        }
        chomp;
        $H{$_} |= $set;
        printf("\t>>\t%s: %s -> %d\n",$ARGV,$_,$H{$_}) if ($debug);
}

$member = 2*$set - 1 if ($member == 0);
printf("\t>>\tmember = %d\n",$member) if ($debug);
for my $i (sort keys %H) {
        printf("%s\n",$i) if ($H{$i} == $member);
}

Above Perl scripts does not need sorted input files, as it stores all records of the files in memory, in a hash. It uses a bitmask as a set. For example, mycomm -s2 F1 F2 prints only those records, which are only in file F2 but not in F1.

Parallelization and CPU Cache Overflow

In the post Rewriting Perl to plain C the runtime of the serial runs were reported. As expected the C program was a lot faster than the Perl script. Now running programs in parallel showed two unexpected behaviours: (1) more parallelizations can degrade runtime, and (2) running unoptimized programs can be faster.

See also CPU Usage Time Is Dependant on Load.

In the following we use the C program siriusDynCall and the Perl script siriusDynUpro which was described in above mentioned post. The program or scripts reads roughly 3GB of data. Before starting the program or script all this data has been already read into memory by using something like wc or grep.

1. AMD Processor. Running 8 parallel instances, s=size=8, p=partition=1(1)8:

for i in 1 2 3 4 5 6 7 8; do time siriusDynCall -p$i -s8 * > ../resultCp$i & done
real 50.85s
user 50.01s
sys 0

Merging the results with the sort command takes a negligible amount of time

sort -m -t, -k3.1 resultCp* > resultCmerged

Best results are obtained when running just s=4 instances in parallel:

$ for i in 1 2 3 4 ; do /bin/time -p siriusDynCall -p$i -s4 * > ../dyn4413c1p$i & done
real 33.68
user 32.48
sys 1.18

Continue reading

Rewriting Perl to plain C

Perl script was running too slow. Rewriting it in C made it 20 times faster.

1. Problem statement. Analyze call-tree dependency in COBOL programs. There are 77 million lines of COBOL code in ca. 30,000 files. These 30,000 COBOL programs could potentially include 74,000 COPY-books comprising 10 million lines of additional code. COBOL COPY-books are analogous to C header-files. So in total there are around 88 million lines of COBOL code. Just for comparison: the Linux kernel has ca. 20 million lines of code.

COBOL program analysis started with a simple Perl script. This Perl script is less than 200 lines, including comments. This script produced the desired dependency information.

Wading through all this COBOL code took up to 25 minutes in serial mode, and 13 minutes using 4 cores on an HP EliteBook notebook using Intel Skylake i7-6600U clocked 2.8 GHz. It took 36 minutes on an AMD FX-8120 clocked with 3.1 GHz. This execution time was deemed too long to see any changes in the output changing something in the Perl script. All runs are on Arch Linux 4.14.11-1 SMP PREEMPT.

2. Result. Rewriting the Perl script in C resulted in a speed improvement of factor 20 when run in serial mode, i.e., run times are now 110s on one core. It runs in 32s when using 8 cores on an AMD FX-8120. C program uses taylormade hashing routines.
Continue reading