Tag Archives: HTML

Electron. Asynchrony, Modules and C/C++

There are a lot of posts about NodeJS describing innovative solutions about JavaScript on server side. However there is another place where JavaScript, V8 and Chrome HTML engine can be applied. That place is your desktop and Electron is a technology that makes possible to create desktop applications with JavaScript/HTML. I’m sure most of us already have experience in Electron. I’m sure that everybody who is reading have at least one Electron based application on this desktop. And some of that applications are really awesome. For example Visual Studio Code, Slack or Atom the beginning of Electron platform. So I’m a little late with this post, but let me share some of my problems and way to solve them.

What is Electron?

According to documentation Electron is NodeJS and Chromium tunning on the same V8 engine. As result we have something like possibility to run node modules in browser. That will allow to access local file system or use any C/C++ modules.
Any Electron application has at least two processes, one main process and one or more renderer. The main process is pure NodeJS process and entry point of the application. Main process is responsible of creation of windows and every window has it’s own renderer process.

In renderer process you are inside Chrome browser but with power of NodeJS.

You can easy communicate between processes with IPC. Electron gives you very simple API for this.

I think that is more that enough about Electron to have general picture of it. You can try it with this tutorial http://electron.atom.io/docs/tutorial/quick-start/.

Asynchronous operations

It is very hard to imagine modern desktop operation without asynchronous operations. We need to show progress while loading something, we need to perform background calculations and many many more features like these. However Electron is not so good in asynchronous operations.

One of the first idea to do something in async way is to send IPC message to main window and wait for results. You may not believe me if I will say that this may freeze your UI but consider following sample

https://github.com/peleshenko/freezing_node_sample

Clone it run npm install and execute run.sh.

When we are trying to freeze UI with long running loop you can not type in input field but css animation is working and you can close window. But when we are trying to freeze application with long running loop in main process even css animation is freezed and you can not close the window.

Amazing!

You can review corresponding discussion on Electron page on github https://github.com/electron/electron/issues/3363

Short answer:

... a blocking operation in main process is easy to block all other processes... ...it is required to put IO and CPU-bound operations in new threads to avoid this...

So the first lesson learn – forget about main process, live it alone, this process is for Chromium not for you, use your own.

Next idea is to use WebWorkers. Electron supports WebWorkers. But unfortunately in WebWorkers you have no access to NodeJS. You can use usual browser API, or calculate something but no NodeJS. That fact makes WebWorkers useless in Electron because most operations that requires async execution are related to file system or other external things.

So there are only one way start something in parallel thread using JavaScript is child_process.fork. Not very simple but working way.

Alternatively you can create a C/C++ addons and do anything you want inside this addon with C++, start any number of threads and dispatch results to Electron’s message loop with callback. But that is another story…

C/C++ Addons

As you can see addons is not something unusual even simple async operation may require addon creation. Additional reason that may enforce you to implement your own addon with C/C++ is quality of existing addons in npm. Some of them are really terrible.

After my last word I think you recall your last attempt of npm module installation in Windows. You had to install specific version of VisualStudio and Python 2.7. My first reaction was “What? The module author is crazy he is going to mix C++, Python and NodeJS. Give me another module…”. But all modules that has C/C++ code inside will require Python and VisualStudio and some times specific version of VisualStudio.

You need all this software to install module because of NodeJS build system node-gyp. That is the only way to include C/C++ addon to your npm module. That is strange thing but if you will take a look a little deeper you will understand the reason. But let me leave this topic for another post.

OK finally you installed VisualStudio and Python and installed module. Starting your Electron application and see following in dev tools console (numbers may differ):

Module version mismatch. Expected 48, got 47"

The source of that problem is that node addon is only compatible with exactly the same node version that this addon has been built for. And you just have built is for your NodeJS but Electron as a little different NodeJS version inside. Doh!!!

To solve the problem you can use electron-rebuild. Follow the instruction and you will be able to rebuild required module for NodeJS version used in Electron.

And the last…

That is it about Electron and problems with it in general. We have more and more projects with Electron in Tesseris Pro. So I hope to share my experience in C/C++ addons and React in Electron in next posts.

And once more. Electron is Chromium + NodeJS, thus it already has NodeJS module system and that system is not so bad. And as for me it looks very strange when people tries to use system.js or require.js or any other JavaScript module system to load modules in Electron. Electron is not a browser! It is NodeJS + browser! Use it as NodeJS!

Some python fun for presentations

The post contains ultimate off-topic, so source code is here 🙂 : https://bitbucket.org/dpeleshenko/mdshow

Except managing managing my company Tesseris Pro and working as a developer I am working with student in one of Kharkiv university. That is a good way to find young talents and prepare them to work in our company.

So I am very often preparing some technical presentations. I have a huge archive of presentations form different technologies. When I prepare my lectures I have to merge this presentations update obsolete data and so on. I was always using MS PowerPoint. But reworking presentation by drag’n’drop is terrible. Additionally I have problems with different slide design. So I am completely unsatisfied with existing mouse driven presentation software.

As a developer I prefer to write everything problem oriented language. First idea was HTML… but it’s too complex… too many letters. LaTex, god but too complex too. I have no math symbols and other similar things in my presentation. Much better is Markdown. After that presentation done with markdown become my dream. And one of important thing was to have every slide in separate file and have separate file for general presentation layout and design. There are some open source solution but none of them was good enough for me. So I decided to make it myself.

My first idea was to convert markdown to HTML and render HTML with CEF. And taking into account that I have another dream – to develop cross platform desktop applications technology with Python business logic and HTML/CSS/JS presentation. I selected python and CEF as main technology stack. But unfortunately all CEF binding for python are terrible. I spent whole day trying to run demo code without any success.

After some additional research I have found very good library to statically render HTML/CSS – http://weasyprint.org/ That was completely enough for me. You can find a python script with about 120 lines to show presentations based on directory with markdown files here: https://bitbucket.org/dpeleshenko/mdshow

To run this code you will need to install following:

I have tested everything in Ubuntu 15.10 but everything should work in any Linux, Mac or Windows. Maybe some fine-tuning with GTK will be required.

Editing code dirrectly in browser (WebKit)

May be some people already know this feature, but I was discovered it only today. A lot of JS debugging bring some benefits and today I found possibility to edit code and save it on disk with Google Chrome (or any other WebKit based browser). Hope this will simplify some debugging tasks for you. See how to enable this feature bellow:

  1. Add folders(s) with your source code to browser’s workspace
    Add folder to  Workspace

  2. Allow browser to access file system (I have Ukrainian browser in English you have to click “Allow”)
    Allow access to file system

  3. Select file and map it on file system resource
    Map to file system

  4. Select resource from mapping from your workspace (added in first step)
    Select file from workspace

  5. Don’t forget to restart dev tool

  6. Edit your file in browser

  7. See changes in VisualStudio or any other IDE
    See changes in your VS

Hope this help you save some time and save Atl and Tab keys on your keyboard 😉

JavaScript and different ***Script

Today I’ll just share my opinion so, no useful information bellow 🙂

There are a lot of discussions around TypeScript, CoffeeScript and other languages translated to JS. Let me add several words to this “holly war”:) I was codding for a couple of months with TypeScript. It was complex UI with Knockout.js and Durandal. Yesterday I switched to pure JavaScript task. I have to help on other our project and I have to write very simple JS code, with just several functions and some calculations.
Just imagine! My performance with JavaScript is at least twice lower than with TypeScript.
Why?
Because “continuous refactoring”, one of key principles to get good code quality, not just good looking code but stable code, is very problematic with JS. You have to check everything (variable names, functions, scopes, etc.) when you changed something in existing code.
My opinion is that Human was not designed to handle so stupid tasks 🙂 Let machines do their work use languages with checks on compilation. And use smart IDE of course 😉
Be Human…

Template project for Node.js with Express

Overview

If you are building node.js applications you may need some template project to start quickly and do not perform simple configuration every time. This configuration may contain access to MongoDB, security infrastructure and so on. My colleague Anton Dimkov has committed his template on Github. Also the template can be installed with npm.

Features

  • Configured simple routes for express
  • Integrated MongoDB access with mongoose
  • Integrated security with password hashing based on bcrypt-nodejs
  • Login page
  • Session management with express-session
  • Logging with winston
  • Integrated Angular and simple SPA front-end structure
  • Bootstrap styles integrated

Installation and configuration

  1. Install Node.js according to instructions at https://nodejs.org/
  2. Install and run MongoDB according instructions at http://www.mongodb.org
  3. Download code from git hub
  4. Install all required modules by runing npm install in terminal
  5. Run application with node www/bin
  6. Open http://localhost:9090/ in web-browser

Try it it’s simple way to start using Node.JS correctly or to start a project.

Fornt-end with Knockout.js, require.js and TypeScript

Let’s talk about how to correctly organize front-end with Knockout.js require.js and TypeScript.

The problem

If we will read TypeScript handbook we will find a lot information about how to load modules with AMD and require.js, and everywhere in samples we will find something like this

    import module=require('./module');

But in real application we always have some folder structure to keep the files organized, we are using different package managers and so on, thus in most cases import should look like

    import ko=require('./node_modules/knockout/build/output/knockout-latest')

Unfortunately for some unknown reason this is not working with TypeScript, at least with versions 1.3 and 1.4. Really, why current folder path is working but more complex path is not? We have to deal somehow with this.

An the only way is to use import ko=require(knockout) instead of full path to knockout.

In this post I will describe the way how to build HTML application with MS VisualStudio and I will use node package manger to load all the libraries, but the same idea will work for nuget or any other package managed and IDE.

Application structure

  • node_modules
    • knockout
      • build
        • output
          • knockout-latest.js
    • requirejs
      • require.js
    • moment
      • moment.js
  • typings
    • knockout
      • knockout.d.ts
    • moment
      • moment.d.ts
  • config.js
  • application.ts
  • mainViewmodel.ts
  • bindings.ts
  • index.html

Require.js enabled JavaScript (or TypeScript) application should start with single “ tag in html. In our case it looks like:

    <script data-main='config.js' src='node_modules/requirejs/require.js'></script>

This config.js is the only JavaScript file, all other logic is done in TypeScript. May be there is some way to write it on TypeScript, but I’m not sure that it makes any sense, because you have to do JS specific low level things here. The config.js looks like following:

    require.config({
        baseUrl: "",
        paths: {
            knockout: "./node_modules/knockout/build/output/knockout-latest",
            moment: "./node_modules/moment/moment"
        }
    });

    define(["require", "exports", 'application'], function (require, exports, app) {
        app.Application.instance = new app.Application();
    });

First of all in this file we are configuring require.js to make it understand where to search for libraries. We will load our index.html from file system and of course in real app you should not use folder structure but think about URLs. Please note that you should not specify file extension.

Now require.js will understand how to load knockout. But this will tell nothing our TypeScript compiler and compiler will report errors about undefined module.

To fix this problem with compiler simply add corresponding typings to the project. Now TypeScript will build everything without errors. Please note that in this case TypeScript will not verify correctness of path to modules because it can’t determine the real URL structure of the application. That may be the reason why complex path is not working in import.

Note: don’t forget to switch TypeScript module type to AMD (Asynchronous Module Definition). This will conflict with node.js and next time I will explain how to deal with node.js and AMD.

Application startup

Our application entry point (after config.js) is application.ts file with following content:

    import vm = require('mainViewModel');
    import ko = require('knockout');
    import bindings = require('bindings');

    export class Application{
        public static instance: Application;

        constructor(){
            bindings.register();
            ko.applyBindings(new vm.mainViewModel());
        }
    }

Here we load module(s) (as dependency) with all custom bindings, create main view model and apply it to whole page.

Note that we don’t need to specify path to bindings and mainViewModel in config.js because they are located at the same directory. You can use more complex structure and everything will work with TypeScript just don’t forget to explain require.js how to find all your modules.

Custom bindings

Custom binding are wrapped in single module and can be loaded as any other module. Binding handlers will be registered with bindings.register() call. This can be done with following content of bindings.ts:

    import ko = require("knockout")
    import moment = require("moment")

    export function register(): void {

        ko.expressionRewriting["_twoWayBindings"].datevalue = true;

        var formatValue = function (value, format) {
            format = ko.unwrap(format);
            if (format == null) {
                format = "DD.MM.YYYY";
            }
            return moment(ko.unwrap(value).toString()).format(format);
        }

        ko.bindingHandlers["datevalue"] = {
            init: function (element: HTMLInputElement, valueAccessor, allBindings, viewModel) {
                element.value = formatValue(valueAccessor(), allBindings.get("dateFormat"));

                element.addEventListener("change", function (event) {
                    var dateValue: any
                        = moment(element.value, ko.unwrap(allBindings.get("dateFormat")))
                            .format("YYYY-MM-DD") + "T00:00:00";

                    if (ko.unwrap(valueAccessor()) instanceof Date) {
                        dateValue = new Date(dateValue);
                    }

                    if (ko.isObservable(valueAccessor())) {
                        valueAccessor()(dateValue);
                    }
                    else {
                        allBindings()._ko_property_writers.datevalue(dateValue);
                    }
                });
            },
            update: function (element: HTMLInputElement, valueAccessor, allBindings) {
                element.value = formatValue(valueAccessor(), allBindings.get("dateFormat"));
            }
        }
    }

Here we create very useful datevalue binding, which allows to edit and display dates as string in specific format. This binding is able to work with observables and flat values and store date in JSON compatible format or Date, depending on initial value of bound property. This binding contains some knockout and TypeScript tricks like ko.expressionRewriting["_twoWayBindings"].datevalue = true and allBindings()._ko_property_writers.datevalue(dateValue) but let’s talk in next blog posts about these tricks.

ViewModel

Nothing special just usual view model organized as module

    import ko = require('knockout');

    export class mainViewModel{

        constructor(){
        }

        public name = ko.observable("John Doe");
        public birthday = ko.observable("1983-01-01");
    }

Conclusion

Everybody are waiting for ECMASript 6 support in all browsers with all sweet things like classes, arrows, modules and so on. Life is too short to wait – let’s use TypeScript today! I’ve tested it in big project and yes sometime it looks a little raw but it’s working and make our life easier with type check and better intellisense.

Knockout.js components. What they are and what they are not

What are components?

Some time ago Knockout.js team released new feature – components. This feature allows developer to build some custom components that will have it’s own view and logic. Registration of component looks almost like binding registration

ko.components.register('mywidget', {
    viewModel: function(params) {
        //Define view model here
        this.title = ko.observable("Hello from component!!!");
    },
    template: '<div data-bind="text: title"></div>'
});

The example below looks very similar to definition of user control in technologies like WPF/Silverlight or even WinForms. We have template to define view of element and view-model to define logic.

Most interesting (for me personally) is usage of these components as custom elements – custom HTML tags. after registrations of my widget in previous examaple you can write following in your HTML code:

<mywidget></mywidget>

Brief description (skip it if you already familiar with components)

And this HTML tag will be replaced with template of the component with applied component view model.

Template can be defined in following way:

  • With existing element id. Template element can be any existing element div or template or anything else.
template: { element: 'my-component-template'}

Element content (only children of element) will be cloned to place where you apply your custom element

  • With exiting element instance
template: { element: getElementById('...') }
  • Directly with string of markup
template: '<div data-bind="text: title"></div>'
  • With array of element instances (elements will be add sequentially)

  • With AMD (Asynchronous Module Definition)

template: { require: 'text!some-template.html' }

Require.js or any other AMD module loader can be used. See https://github.com/amdjs/amdjs-api/blob/master/AMD.md for details.

For view-model configuration you can use following:

  • Constructor function
viewModel: function(params) {
    //Define view model here
    this.title = ko.observable("Hello from component!!!");
}
  • Existing instance of view model
viewModel: { instance: viewModelInstance }
  • Factory function
viewModel: {
              createViewModel: function(params, componentInfo) { 
                                  return new ViewModel(params); 
                               }
           }

Here we have additional parameter componentInfo. This parameter allow us to get access to our custom element with componentInfo.element. But unfortunately we can’t get access to this element before template is applied and analyze it as it was initially added to the document. I’ll describe why I’ve said unfortunately a little later

  • Load view model through AMD
viewModel: { require: 'some/module/name' }
  • Specify whole the component as single AMD module
define(['knockout'], function(ko) {
    return {
        viewModel: function(params) {
           this.title = ko.observable("Hello from component!!!");
        },
        template: '<div data-bind="text: title"></div>'
    };
});

And register it with

ko.components.register('my-component', { require: 'some/module' });

What components are not?

Let’s assume we would like to build a component for bootstrap button with popover. And we would like to open this popover when button is clicked and when another button inside popover is clicked we would like to call some handler in view-model. Something like button with confirmation. And we would like to add custom confirmation template with elements bound to view model.

donate-btn

It would be nice to have component with following syntax

<popover text="Donate" 
		 data-bind="command: makeDonation"
		 title="Enter amount of donation">
	<input class='form-control text-right' type='text' 
               data-bind='value: donationAmount' />
</popover>

But unfortunately it’s not possible. There is no way to read HTML content of the component applied as custom HTML tag, because everything view-model factory, view-model constructor and all other functions are called when template is applied to component and template is required parameter.
Thus you can’t build custom controls with templates inside. Only one possible option is to specify template id as parameter of the your custom control.

<template id='donate-template'>
    <input class='form-control text-right' type='text' 
           data-bind='value: donationAmount' />
</template>

<popover text="Donate" 
		 title="Enter amount of donation" 
		 data-bind="command: makeDonation" 
		 template="donate-template"></popover>

Or use usual binding instead of component to specify template control

<div class="btn btn-xs btn-primary">
   <div data-bind="popover: {title:'Enter ammount', command: makeDonation}"
                   class="hidden">
	 <input class='form-control text-right' type='text' 
                data-bind='value: donationAmount' />
   </div>
   Donate
</div>

More or less equivalent code but imagine how useful this “inline templateing” can be for controls like this http://grid.tesseris.com/Home/Documentation#!/General/general

Let’s hope for the future versions…