Posted by Gizra on Tue, 05/22/2018 - 08:00

This is going to be a simple exercise to create a decoupled site using Drupal 8 as the backend and an Elm app in the frontend. I pursue two goals with this:

  • Evaluate how easy it will be to use Drupal 8 to create a restful backend.
  • Show a little bit how to set up a simple project with Elm.

We will implement a very simple functionality. On the backend, just a feed of blog posts with no authentication. On the frontend, we will have a list of blog posts and a page to visualize each post.

Our first step will be the backend.

Continue reading…

Posted by Alex Korban on Mon, 05/07/2018 - 03:00

New release is out today. Now it allows you to:

Create subsections with their own posts
Create Index.elm pages which turn into index pages, both at the root level and at subsection level. For example, Pages/Index.elm becomes the page at / in contrast to Pages/Projects.elm which becomes /projects. At subsection level, Pages/Elm/Index.elm becomes the page at /elm, while Pages/Elm/Projects.elm becomes /elm/projects.
Use the postprocessing copy step to give pages aliases (eg /projects can be made ...

Posted by Alex Korban on Thu, 05/03/2018 - 03:00

I've been using Elm to generate this site for a few months. It's still highly experimental and incomplete but I hope that other Elm enthusiasts may find it useful.

At the moment, it works for my needs and allows me to generate this site with a single command, but it requires manual intervention if I'm doing things like deleting pages or adding new Elm dependencies, and I'm sure there are many use cases it doesn't address.

I think I have something barebones but useful so I published it on NPM ...

Posted by Alex Korban on Fri, 04/13/2018 - 03:00

There are a number of posts about decoding recursive JSON in Elm, but I still found it was a slightly tricky task, particularly because the way the Elm compiler deals with recursion requires a workaround in the decoder code to avoid rather confusing runtime errors. In this post, I'd like to tell the story of implementing recursive decoders.

The path to a recursive decoder

The basic structure of the JSON we need to decode is like this (this happens to be the PostgreSQL EXPLAIN command output):...

Posted by Alex Korban on Fri, 04/13/2018 - 03:00

There are a number of posts about decoding recursive JSON in Elm, but I still found it was a slightly tricky task, particularly because the way the Elm compiler deals with recursion requires a workaround in the decoder code to avoid rather confusing runtime errors. In this post, I'd like to tell the story of implementing recursive decoders.

The path to a recursive decoder

The basic structure of the JSON we need to decode is like this (this happens to be the PostgreSQL EXPLAIN command output):...

Posted by Alex Korban on Thu, 03/29/2018 - 03:00

I've written a static site generator to produce my site (yaks have to be shaved, after all). Unlike thousands of other static site generators out there, mine allows me to write pages in Elm, specifically because I wanted to be able to use style-elements and get away from CSS.

I'm using eeue56/elm-static-html to convert Elm to plain HTML, and I'm about to switch to eeue56/elm-static-html-lib (thanks to Noah for his many contributions to Elm!).

But how does it work? Normally, our views produce ...

Posted by Alex Korban on Thu, 03/29/2018 - 03:00

I've written a static site generator to produce my site (yaks have to be shaved, after all). Unlike thousands of other static site generators out there, mine allows me to write pages in Elm, specifically because I wanted to be able to use style-elements and get away from CSS.

I'm using eeue56/elm-static-html to convert Elm to plain HTML, and I'm about to switch to eeue56/elm-static-html-lib (thanks to Noah for his many contributions to Elm!).

But how does it work? Normally, our views produce ...

Posted by Alex Korban on Fri, 03/23/2018 - 03:00

I've seen a lot of discussion and concern around Elm development activity recently. This comment on Hacker News is a good example: "Most packages I use and Elm itself is weeks and months old, makes me worried that it's just stale". Someone else paraphrased their boss's opinion as "Looks like it's dead now, no releases in over a year".

The concern is that this sentiment either puts people off trying Elm, or it makes them consider not using Elm anymore, or it stops them from recommending Elm to ...

Posted by Alex Korban on Fri, 03/23/2018 - 03:00

I've seen a lot of discussion and concern around Elm development activity recently. This comment on Hacker News is a good example: "Most packages I use and Elm itself is weeks and months old, makes me worried that it's just stale". Someone else paraphrased their boss's opinion as "Looks like it's dead now, no releases in over a year".

The concern is that this sentiment either puts people off trying Elm, or it makes them consider not using Elm anymore, or it stops them from recommending Elm to ...

Posted by Alex Korban on Fri, 03/16/2018 - 02:00

Union types are everywhere in Elm, but when you're starting to learn the language, they can be a bit unfamiliar and confusing, because they don't really exist in other languages like JavaScript or C#.

The main question I've seen people ask is: how do I extract a value from a union type?

Suppose you have a union type like this:

type UserId
= Id Int
| Uuid String

A UserId value can contain either an Int or a String. The solution for getting to those values is simple because Elm provid...

Posted by Alex Korban on Fri, 03/16/2018 - 02:00

Union types are everywhere in Elm, but when you're starting to learn the language, they can be a bit unfamiliar and confusing, because they don't really exist in other languages like JavaScript or C#.

The main question I've seen people ask is: how do I extract a value from a union type?

Suppose you have a union type like this:

type UserId
= Id Int
| Uuid String

A UserId value can contain either an Int or a String. The solution for getting to those values is simple because Elm provid...

Posted by Alex Korban on Thu, 03/15/2018 - 02:00

A question I've seen quite often from people new to Elm is "How do I read types like Html Msg"? When the function signature is view : Model -> Html Msg, does view return some kind of function which takes a Msg argument? Or does it return a function that returns a Msg? Or is Html instead some kind of type decorator that extends Msg?

When you start out with Elm, this stuff can be confusing - I know I stumbled over it for a while.

To understand what's going on, a useful analogy to recall is the ...

Posted by Alex Korban on Thu, 03/15/2018 - 02:00

A question I've seen quite often from people new to Elm is "How do I read types like Html Msg"? When the function signature is view : Model -> Html Msg, does view return some kind of function which takes a Msg argument? Or does it return a function that returns a Msg? Or is Html instead some kind of type decorator that extends Msg?

When you start out with Elm, this stuff can be confusing - I know I stumbled over it for a while.

To understand what's going on, a useful analogy to recall is the ...

Posted by Gizra on Thu, 03/08/2018 - 08:00

Testing at Gizra
We have covered the subject of automatic tests quite a lot here at Gizra and I’m sure that there are a lot of posts and articles covering the subject. However, I’m having a bizarre experience lately while writing new features that I had to share it with you folks. Keep in mind that I don’t write a lot of blog posts and I always feared following up on my previous smashing hit Bootstrap custom breakpoint :)

So the experience I want to talk about was that of insecurity, which was quite odd for me considering I’m always proud of my code until @amitaibu reviews it and smashes my ego to pieces (which I learned to love).

Continue reading…

Posted by Gizra on Fri, 02/16/2018 - 08:00

Chances are that you already use Travis or another cool CI to execute your tests, and everyone politely waits for the CI checks before even thinking about merging, right? More likely, waiting your turn becomes a pain and you click on the merge: it’s a trivial change and you need it now. If this happens often, then it’s the responsibility of those who worked on those scripts that Travis crunches to make some changes. There are some trivial and not so trivial options to make the team always be willing to wait for the completion.

This blog post is for you if you have a project with Travis integration, and you’d like to maintain and optimize it, or just curious what’s possible. Users of other CI tools, keep reading, many areas may apply in your case too.

Unlike other performance optimization areas, doing before-after benchmarks is not so crucial, as Travis mostly collects the data, you just have to make sure to do the math and present the numbers proudly.

Caching

To start, if your .travis.yml lacks the cache: directive, then you might start in the easiest place: caching dependencies. For a Drupal-based project, it’s a good idea to think about caching all the modules and libraries that must be downloaded to build the project (it uses a buildsystem, doesn’t it?). So even a variant of:

cache:
directories:
- $HOME/.composer/cache/files

or for Drush

cache:
directories:
- $HOME/.drush/cache

It’s explained well in the verbose documentation at Travis-ci.com. Before your script is executed, Travis populates the cache directories automatically from a successful previous build. If your project has only a few packages, it won’t help much, and actually it can make things even slower. What’s critical is that we need to cache slow-to-generate, easy-to-download materials. Caching a large ZIP file would not make sense for example, caching many small ones from multiple origin servers would be more beneficial.

From this point, you could just read the standard documentation instead of this blog post, but we also have icing on the cake for you. A Drupal installation can take several minutes, initializing all the modules, executing the logic of the install profile and so on. Travis is kind enough to provide a bird’s-eye view on what eats up build time:

Execution speed measurements built in the log

Mind the bottleneck when making a decision on what to cache and how.

For us, it means cache of the installed, initialized Drupal database and the full document root. Cache invalidation is hard, we can’t change that, but it turned out to be a good compromise between complexity and execution speed gain, check our examples:

Do your homework and cache what’s the most resource-consuming to generate, SQL database, built source code or compiled binary, Travis is here to assist with that.

Software Versions

There are two reasons to pay attention to software versions.

Use Pre-installed Versions
Travis uses containers of different distributions, let’s say you use trusty, the default one these days, then if you choose PHP 7.0.7, it’s pre-installled, in case of 7.1, it’s needed to fetch separately and that takes time for every single build. When you have production constraints, that’s almost certainly more important to match, but in some cases, using the pre-installed version can speed things up.

And moreover, let’s say you prefer MariaDB over MySQL, then do not sudo and start to install it with the package manager, as there is the add-on system to make it available. The same goes for Google Chrome, and so on.
Stick to what’s inside the image already if you can. Exploit that possibility of what Travis can fetch via the YML definition!

Use the Latest and (or) Greatest

If you ever read an article about the performance gain from migrating to PHP 7, you sense the importance of selecting the versions carefully. If your build is PHP-execution heavy, fetching PHP 7.2 (it’s another leap, but mind the backward incompatibilities) could totally make sense and it’s as easy as can be after making your code compatible:

language: php
php:
- '7.2'

Almost certainly, a similar thing could be written about Node.js, or relational databases, etc. If you know what’s the bottleneck in your build and find the best performing versions – newer or older – it will improve your speed. Does that conflict with the previous point about pre-installed versions? Not really, just measure which one helps your build the most!

Make it Parallel

When a Travis job is running, 2 cores and 4 GBytes of RAM is available – that’s something to rely on! Downloading packages should happen in parallel. drush make, gulp and other tools like that might use it out of the box: check your parameters and configfiles. However, on the higher level, let’s say you’d like to execute a unit test and a browser-based test, as well. You can ask Travis to spin up two (or more) containers concurrently. In the first, you can install the unit testing dependencies and execute it; then the second one can take care of only the functional test. We have a fine-grained example of this approach in our Drupal-Elm Starter, where 7 containers are used for various testing and linting. In addition to the great execution speed reduction, the benefit is that the result is also more fine-grained, instead of having a single boolean value, just by checking the build, you have an overview what can be broken.

All in all, it’s a warm fuzzy feeling that Travis is happy to create so many containers for your humble project:

If it's independent, no need to serialize the execution

Utilize RAM

The available memory is currently between 4 and 7.5 GBytes , depending on the configuration, and it should be used as much as possible. One example could be to move the database main working directory to a memory-based filesystem. For many simpler projects, that’s absolutely doable and at least for Drupal, a solid speedup. Needless to say, we have an example and on client projects, we saw 15-30% improvement at SimpleTest execution. For traditional RMDBS, you can give it a try. If your DB cannot fit in memory, you can still ask InnoDB to fill memory.

Think about your use case – even moving the whole document root there could be legitimate. Also if you need to compile a source code, doing it there makes sense as well.

Build Your Own Docker Image

If your project is really exotic or a legacy one, it potentially makes sense to maintain your own Docker image and then download and execute it in Travis. We did it in the past and then converted. Maintaining your image means recurring effort, fighting with outdated versions, unavailable dependencies, that’s what to expect. Still, even it could be a type of performance optimization if you have lots of software dependencies that are hard to install on the current Travis container images.

+1 - Debug with Ease

To work on various improvements in the Travis integration for your projects, it’s a must to spot issues quickly. What worked on localhost, might or might not work on Travis – and you should know the root cause quickly.

In the past, we propagated video recording, now I’d recommend something else. You have a web application, for all the backend errors, there’s a tool to access the logs, at Drupal, you can use Drush. But what about the frontend? Headless Chrome is neat, it has built-in debugging capability, the best of which is that you can break out of the box using Ngrok. Without any X11 forwarding (which is not available) or a local hack to try to mimic Travis, you can play with your app running in the Travis environment. All you need to do is to execute a Debug build, execute the installation part (travis_run_before_install, travis_run_install, travis_run_before_script), start Headless Chrome (google-chrome --headless --remote-debugging-port=9222), download Ngrok, start a tunnel (ngrok http 9222), visit the exposed URL from your local Chrome and have fun with inspection, debugger console, and more.

Takeaway

Working on such improvements has benefits of many kinds. The entire development team can enjoy the shorter queues and faster merges, and you can go ahead and apply part of the enhancements to your local environment, especially if you dig deep into database performance optimization and make the things parallel. And even more, clients love to hear that you are going to speed up their sites, as this mindset should be also used at production.

Continue reading…