Gulp Watch to build only changed TypeScript

Performance when building TypeScript can be crutial if you are working on a big projects. If you are using watcher to compile when any TypeScript file change and use Gulp Watch to compile every TypeScript file, then you will have a huge performance hit. It means that if you change 1 file that you may have to build thousand of them. The following code is the lazy approach that build every TypeScript file if one changed:

gulp.watch(paths.typescript_in + '**/*.ts', ['build', 'tsreload']);

This script watch for TypeScript files to be changed, if it does, run the build task and reload the browser. The problem is that the build task build all TypeScript. To remedy that situation, we want TypeScript to only build the changed file. For that, you will need a new Gulp package called “gulp-cached” that you install as a dev dependency.

npm install gulp-cached --save-dev

Inside your gulpfile.js, you need to access the module:

const changed = require('gulp-cached');

And finally, you need to use the “on change” event after the watch, and remove the tasks’s dependencies.

    gulp.watch("app/scripts/**/*.ts").on("change", function() {
        var compilationResults = gulp.src("app/scripts/**/*.ts")
            .pipe(changed("./deploy/output"))
            .pipe(sourcemaps.init())
            .pipe(tsProject())
        compilationResults.dts.pipe(gulp.dest("./deploy/output"));
        compilationResults.js
            .pipe(sourcemaps.write('.'))
            .pipe(gulp.dest("./deploy/output"))
            .pipe(connect.reload());
    });

The main change is that we pipe through the changed call the destination. This pipe, once ran once, will keep data about if the file change. If this one change, it will go down the pipeline. Otherwise, it will be filtered out. It means that the first time a TypeScript file change, the watch will build everything. After, it will only filter all the source down the changed file. The reload is by calling directly connect.reload() at the end. This is a huge performance boost for your as a developer because you will be able, what ever the size of the project you are working on, to build under 1 sec every change you do. Having a rapid window between you save your file and the time you can see your change in your browser is critical to ship fast code. With this library that act a cache, you can benefit of filtering out the noise that doesn’t change and concentrate your computer to build only what is required.

Compiling TypeScript for a specific folder to increase build performance

If you are using Gulp to build your TypeScript you may use the default configuration that is used in a lot of website which is letter the tsconfig.json to handle what to include and having Gulp to use Gulp-TypeScript to read the tsconfig.json file. However, if you want to build just a portion of the TypeScript, let say a single folder, you will be out of luck.

So the idea is to not use this kind of configuration in tsconfig.json

{
  "compilerOptions": {
    "sourceMap": true,
    "target": "es6",
    "module": "amd",
    "outDir": "./deploy/output",
    "types": [
      "jquery",
      "requirejs",
      "lodash",
      "reflect-metadata"
    ],
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true
  },
  "include": [
    "app/scripts/**/*"
  ],
  "exclude": [
    "node_modules",
    "**/*.spec.ts"
  ]
}

and not this task in Gulp task code:

gulp.task("build", () => {
    const r = "./app/output";
    var compilationResults = tsProject.src()
        .pipe(sourcemaps.init())
        .pipe(tsProject())
    compilationResults.dts.pipe(gulp.dest(r));
    return compilationResults.js
        .pipe(sourcemaps.write('.'))
        .pipe(gulp.dest(r));
});

But to use this tsconfig.json:

{
  "compilerOptions": {
    "sourceMap": true,
    "target": "es6",
    "module": "amd",
    "outDir": "./deploy/output",
    "types": [
      "jquery",
      "requirejs",
      "lodash",
      "reflect-metadata"
    ],
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true
  },
  "exclude": [
    "node_modules",
    "**/*.spec.ts"
  ]
}

and this Gulp task code:

gulp.task("build", () => {
    const outFolder  = "./app/output";
    var compilationResults = gulp.src("app/scripts/**/*.ts")
        .pipe(sourcemaps.init())
        .pipe(tsProject())
    compilationResults.dts.pipe(gulp.dest(outFolder));
    return compilationResults.js
        .pipe(sourcemaps.write('.'))
        .pipe(gulp.dest(outFolder));
});

The whole idea is that you can move the include files outside TypeScript configuration file, but inject the files from Gulp. So far, everything is build from the root of the app/script folder, but you could define a new task that take a sub folder like the following code:

gulp.task("buildgeneral", () => {
    const outFolder = "deploy/output/general";
    var compilationResults = gulp.src("app/scripts/general/*.ts")
        .pipe(sourcemaps.init())
        .pipe(tsProject())
    compilationResults.dts.pipe(gulp.dest(outFolder));
    return compilationResults.js
        .pipe(sourcemaps.write('.'))
        .pipe(gulp.dest(outFolder));
});

This is very interesting if you have a project with thousand files. Instead of building the whole project every time, you can just build the file or the folder that the file reside.

Visual Studio Code with NPM and TypeScript (Part 8 : SASS & CSS)

Transforming SCSS into CSS is a matter of using less than 5 lines of code. NPM has a package that you can use with Gulp. In less than 5 minutes, you can have SCSS configured to compile all your files into CSS.

The first step is to get the gulp sass package. You need to install the dependency has development since you won’t need it at the browser level.

npm install gulp-sass --save-dev

The next step is to create a step that will take all you scss files, pipe the files into the sass compiler and pipe the output into your output folder, where you will references them from your HTML file to serve the style. Here is the Gulp task.

gulp.task('scss', function() {
  gulp.src('sass/**/*.scss')
        .pipe(sass().on('error', sass.logError))
        .pipe(gulp.dest('./css/'));
});

You can find all changes for SCSS inside this commit[1] for the project we are building up. You may see some differences in the commit compared to this blog post. For example, instead of relying on string directly inside the task for path, I opted to use a constant. It’s always better to have in a centralized way in case of change or in case that it can be reused in another task.

Another change that you might consider is to add this new created task in the dependency of tasks for the build task or the watcher.

Here is a screenshot of the output where the SCSS generates CSS that modify the HTML.

SCSS:

body{
    font-size:36px;
    span{
        font-size:6px;
    }
}

CSS:

body {
  font-size: 36px; 
}
body span {
    font-size: 6px; 
}

HTML Output:

[1]:https://github.com/MrDesjardins/TypescriptNpmGulp/commit/02410d535a12cb48b167ff45b18970d312776270

Visual Studio Code with NPM and TypeScript (Part 7 : watcher & auto-refresh)

Developing website front-end gives some habit to have a fast preview of the modification we do in the code. Mostly because HTML, CSS and JavaScript at the base doesn’t require any compilation. Simply saving the file and refreshing the browser was enough to see the new code working in the browser. However, with TypeScript, we need to compile to have a new release of the output JavaScript. To do so, we need something to watch all TypeScript files, compile and refresh the browser.

This basic example is not very optimized for a big code base because this new Gulp task will watch all TypeScript file and if one change, will rebuild every TypeScript. In a more enterprise scenario or big project, we would have smarter watch to work on branch of the code which would result to a fast build of file or section who need to get updated.

Creating a task is done like usual. The trick is to have a dependent task at the “gulp.watch”. This watch doesn’t require any dependency to npm package, it’s there with Gulp. It takes file pattern to know which files to look for changes and an array of other task to do when something change.

gulp.task('watch', function() {
    gulp.watch(paths.typescript_in + '/*.ts', ['build']);
});

The next step is to have the browser reload itself automatically. We were using ExpressJs, but we will use something else from that point. We will use gulp-connect. It’s a very light-weight server with live reload integrated.

npm install gulp-connect --save-dev

We can delete the server.js file we had before and add the following gulp task into gulpfile.js

gulp.task('watch', function() {
    gulp.watch(paths.typescript_in + '/*.ts', ['build']);
});

gulp.task('server', function() {
  connect.server({
    root: 'output',
    livereload: true,
    port: 8080
  });
});

gulp.task('go', ["server", "watch"], function() {

});

We also need to do two changes to actual task. The first one is once we are done building TypeScript, we notify the connector to reload. The second one is to move the index.html, boot page, in the output folder since we will run the server from this server.

gulp.task("build", function () {
   //...
        .pipe(connect.reload());
});

and

gulp.task("copy", function () {
   //...
     gulp.src("./src/index.html").pipe(gulp.dest(paths.typescript_out));
});

From that point, anytime we run the server, by executing : gulp go, the server will execute and wait for any changes since we have a dependency to watch who is looking for changes to compile. Once compilation is over, the connect call reload which is using a websocket to communicate to the webpage to reload.

You can find the whole source code in GitHub : https://github.com/MrDesjardins/TypescriptNpmGulp

Visual Studio Code with NPM and TypeScript (Part 6 : Lazy-loading)

To have the fastest website possible you must be wise to not load unnecessary JavaScript. In some scenario, it is the good strategy to not have everything in a bundle. For example, if a kind of action require different JavaScript files but that this one is not in the main scenario then it’s better to lazy-load this module. This article will explain how to lazy-load TypeScript module.

Let’s fake the scenario that we want to load a module 2 seconds after a method is called:

export class ClassB {
    public method1(): void {
        console.log("ClassB>method1");
        setTimeout(() => {
           //Use code from fileToLazyLoad.ts
        }, 2000);
    }
}

The way to do it is to use directly the AMD loader instead of using the EcmaScript import. This might not be the case in the future, but this is how to do it at the time I am writing this post. From the previous article about TypeScript, we configured to use AMD (requirejs). To have the TypeScript lazy-loaded, we will need to use require directly in the file. The first step is to add at the top of the file a require to a reference to the type. This won’t load the file at runtime, but let us in development time a reference to the type. The second step is to use requirejs() method at the time you need to module. The third step is to have the types aware in every modules. This last and third step is done by using :

npm install @types/requirejs --save-dev

The first and second step can be seen in this transformed code form the initial snippet:

import foo = require("folder1/fileToLazyLoad");
export class ClassB {
    public method1(): void {
        console.log("ClassB>method1");
        setTimeout(() => {
            requirejs(["folder1/fileToLazyLoad"], (c: typeof foo) => {
                const co = new c.ClassC();
                co.method1();
            });
        }, 2000);
    }
}

You can see that you can load multiple modules since the first parameter is an array. The second is a function that contains the same amount of argument that the length of the array. It’s at this moment that the browser will do a HTTP request to get JavaScript file and execute the code. In that example, the method1() is called once the file is downloaded.

Here is how the browser handles the HTTP request:

Lazy loading should be in your tool belt and be used anytime you can delay unnecessary module on the critical path. Good case are for dialog window, or deep hidden menu options. However, be careful not to degrade the user experience by having the user to wait on every click. Nothing is more annoying than clicking a menu and having a white space for few milliseconds. While not everything needs to be loaded, once the user is having what needed to work, you should eagerly load alternate scenario. At the end, the best strategy is to use lazy-loading at the right moment to have a good balance of performance and user experience.

Visual Studio Code with NPM and TypeScript (Part 5 : TsLint)

Code needs to have some consistency to be able to have a uniform reading later. When your code grows and the number of software developers, it becomes harder to handle it manually. TsLint is a TypeScript linter who is a code analyzer that gives which line needs improvement in term of code readability, maintainability, and functionality errors. TsLint is configurable for your own preferences and can be executed manually, automatically when writing code or as a step in your continuous integration process.

To install TsLint, you need to install the TsLint npm package:

npm install -g tslint typescript --savedev
npm install gulp-tslint --savedev

You have to create a configuration file “tslint.json” at the root of your project. The position of this file is the same as the tsconfig.json or typings.json or gulpfile.js file.

In this file, you can configure the rules you want. A basic configuration is to extend an existing one and override some properties to your taste.

{
  /*
   * Possible values:
   * - the name of a built-in config
   * - the name of an NPM module which has a "main" file that exports a config object
   * - a relative path to a JSON file
   */
  "extends": "tslint:latest",
  "rules": {
    /*
     * Any rules specified here will override those from the base config we are extending.
     */
    "curly": true,
    "triple-equals": true
  },
  "jsRules": {
    /*
     * Any rules specified here will override those from the base config we are extending.
     */
    "curly": true
  },
  "rulesDirectory": [
    /*
     * A list of relative or absolute paths to directories that contain custom rules.
     * See the Custom Rules documentation below for more details.
     */
  ]
}

In the example above, we extend “latest”. You can read the configuration in their official GitHub:
https://github.com/palantir/tslint/tree/master/src/configs

If we want to have the rule to be executed, we need to use TsLint extension for VsCode.

At this point, you may have to reboot VsCode to have the extension to be executed when you type in Visual Studio Code. Once restarted, you should see :

The green lines indicate that the import is missing a semi-colon, that the var should be let/const and that a triple equals should be used.

Next step, is to have a Gulp task that can be executed. It can be a good idea to have this task ran automatically with your task that will build, run test, lint code. To do so, we need to use the gulp-tslint module.

const gulp_tslint = require('gulp-tslint');

The task needs to get all TypeScript files, but skip definition files and node_modules.

gulp.task('tslint', () => {
    return gulp.src(['**/*.ts', '!**/*.d.ts', '!node_modules/**'])
      .pipe(gulp_tslint())
      .pipe(gulp_tslint.report());
});

To use, in the console use : gulp tslint and you will get a report that looks like below:

[20:08:53] Using gulpfile D:\code\tsWithCode1\gulpfile.js
[20:08:53] Starting 'tslint'...
D:/code/tsWithCode1/src/file1.ts[7, 2]: file should end with a newline
D:/code/tsWithCode1/src/file1.ts[4, 1]: Forbidden 'var' keyword, use 'let' or 'const' instead
D:/code/tsWithCode1/src/file1.ts[1, 41]: Missing semicolon
D:/code/tsWithCode1/src/file1.ts[5, 7]: == should be ===

D:/code/tsWithCode1/src/fileToInclude.ts[9, 2]: file should end with a newline
D:/code/tsWithCode1/src/fileToInclude.ts[4, 9]: Calls to 'console.log' are not allowed.
D:/code/tsWithCode1/src/fileToInclude.ts[3, 5]: misplaced opening brace
D:/code/tsWithCode1/src/fileToInclude.ts[5, 13]: Identifier 'div' is never reassigned; use 'const' instead of 'let'.

From this point, you can increase the amount of rules and have a more rigid code or less depending of your need. Do not forget to add into your repository the configuration to be sure that everyone of the team uses the same rules.

Visual Studio Code with NPM and TypeScript (Part 3 : Gulp)

Previously, we setup TypeScript but had two problems. The first one was that we couldn’t move the vendor library and the second that we couldn’t test the generated TypeScript file since we do not have a web server running. We will tackle the first problem by using Gulp. Gulp is like MsBuild, it’s only goal is to help you automate some tasks.

To be able to use Gulp, we need to download gulp. It’s convenient since we can get it from npm by executing the install command.

npm install gulp --save-development
//or
npm install gulp -g

Create a Gulp’s configuration file : gulpfile.js at the root, sibling to TypeScript’s configuration file, and typings’ configuration file. We will need some additional node modules to help us working inside Gulp. You can get the one we will use by typing:

npm install gulp-typescript --save-development
npm install typescript --save-development
npm install del --save-development
npm install gulp-sourcemaps --save-development

This file is pure JavaScript that use Node’s module to execute JavasScript for specific tasks. The task runner is Gulp. The goal is to be able in the console to write:

gulp clean
//or
gulp build

The Gulp file that we will build is simple for the case of learning how to use Gulp to do:

      Clean up previous build file
      Use TypeScript to compile
      Move vendors (third-party) libraries

Before creating these 3 Gulp’s tasks, we need to instanciate some node’s modules that will help us doing the work.

var gulp = require('gulp');
var tsc = require('gulp-typescript');
var del = require('del');
var sourcemaps = require('gulp-sourcemaps');
var tsProject = tsc.createProject('tsconfig.json');

The main one is Gulp which is the runner. This will create entry point that we can invoke by the command line. Gulp-typescript will read our tsconfig.json and use TypeScript to build the code. Del is a library to delete files, which we will use to delete old files. Gulp-sourcemaps is used to create .map.js file, for debugging purpose. Finally the last line just load the TypeScript configuration by using gulp-typescript module that we just loaded.

The next step is to define some constants to help us having a cleaner Gulp code.

var paths = {
    webroot: "./",
    node_modules: "./node_modules/",
    typescript_in: "./src/",
    typescript_out: "output",
    typings: "./typings/",
    typescript_definitions: "./typings/main/**/*.ts"
};
paths.allTypeScript = paths.typescript_in + "**/*.ts";
paths.modulesDestination = paths.webroot + "vendors/";

Here start the fun. Let’s create the first Gulp task to delete previous generated JavaScript file and mapping file from TypeScript. This is required because even if TypeScript will overwrite previous JavaScript generated, if you delete a TypeScript, the JavaScript will hang in the folder which can cause size problem once we bundle all JavaScript.

gulp.task("clean", function (callback) {
    var typeScriptGenFiles = [
        paths.typescript_out + "/**/*.js",
        paths.typescript_out + "/**/*.js.map"
    ];

    del(typeScriptGenFiles, callback);
});

The syntax is the name of the task, followed by an optional depencency array (not present in this example) and a callback function. The purpose of this callback is to notify Gulp that the task is done.

In our simple case, we pass an array of file patterns to a node’s module to delete the file. Nothing complex. Let’s move on to the vendor copy task.

The second task is to copy vendors library. This time, we use Gulp’s functions that will get a list of library, get the code, pipe the result into a new destination.

gulp.task("copy", function () {
    var modulesToMove = {
        "jquery": "jquery/dist/jquery*.{js,map}"
    }

    for (var destinationDir in modulesToMove) {
        gulp.src(paths.node_modules + modulesToMove[destinationDir])
            .pipe(gulp.dest(paths.modulesDestination + destinationDir));
    }
});

Finally, the biggest task : building. This need to create the TypeScript and if needed definition files. We use the tsProject variable that come from the tsconfig.json, get all the source, generate the source map, compile. After, we take all definition file and write everything at the destination path. Then, we do the same with JavaScript file where we write the source map and source in the destination folder. Here how it looks:


gulp.task("build", function () {
    var sourceTsFiles = [paths.typescript_in, paths.typescript_definitions];
    var compilationResults = tsProject.src()
        .pipe(sourcemaps.init())
        .pipe(tsProject())
    compilationResults.dts.pipe(gulp.dest(paths.typescript_out));
    return compilationResults.js
        .pipe(sourcemaps.write('.'))
        .pipe(gulp.dest(paths.typescript_out));
});

So far, we can go in the console and use : gulp clean or gulp copy or gulp build. We saw that we can have a second parameter that is an array of task name, we could have a task that execute these three tasks we just created which will allow us to execute gulp buildall and have the clean, copy and build executed.

gulp.task("buildall", ["clean", "copy", "build"], function (callback){
    callback();
});

Gulp’s tasks could be even more complex, do JavaScript minimization, bundle, etc. The idea is that you need to find existing node modules (or create your own) and use it.