Posted by Gizra on Fri, 01/05/2018 - 08:00

Once you start writing apps (and packages) in Elm, it’s
tempting to avoid the rough-and-tumble world of Javascript as much as possible.
Yet when implementing features for paying clients, it doesn’t always make sense
to take things that already have a Javascript implementation and re-implement
them in pure Elm. In fact, sometimes it isn’t even possible!

Now, Elm has a very fine mechanism for integrating bits of Javascript when
necessary – ports!
Yet ports aren’t always the right answer, and there are several alternatives
which can be useful in certain situations.

For the purposes of this post, I’m going to assume that you’re familiar with
the many cases in which ports work well, and focus instead on a few cases where
you might want to try something else:

  • When you want synchronous answers.
  • When you need some context when you get the answer.
  • When you want to manage parts of the DOM using Javascript.

Continue reading…

Posted by Gizra on Tue, 01/02/2018 - 08:00

I tell my kids all the time that they can’t have both - whether it’s ice cream and cake or pizza and donuts - and they don’t like it. It’s because kids are uncorrupted, and their view of the world is pretty straightforward - usually characterized by a simple question: why not?

And so it goes with web projects:

Stakeholder: I want it to be like [insert billion dollar company]’s site where the options refresh as the user makes choices.

Me: [Thinks to self, “Do you know how many millions of dollars went into that?”] Hmm, well, it’s complicated…

Stakeholder: What do you mean? I’ve seen it in a few places [names other billion dollar companies].

Me: [Gosh, you know, you’re right] Well, I mean, that’s a pretty sophisticated application, and well, your current site is Drupal, and well, Drupal is in fact really great for decoupled solutions, but generally we’d want to redo the whole architecture… and that’s kind of a total rebuild…

Stakeholder: [eyes glazed over] Yeah, we don’t want to do that.

But there’s is a way.

Continue reading…

Posted by Gizra on Mon, 12/25/2017 - 00:00

If you happen to know Brice - my colleague and Gizra’s CEO - you probably have picked up that he doesn’t get rattled too easily. While I find myself developing extremely annoying ticks during stressful situations, Brice is a role model for stoicism.

Combine that with the fact that he knows I dislike speaking on the phone, let alone at 6:53pm, almost two hours after my work day is over, you’d probably understand why I was surprised to get a call from him. “Surprised” as in, immediately getting a stomach ache.

The day I got that call from him was a Sale day. You see, we have this product we’ve developed called ״Circuit Auction״, which allows auction houses to manage their catalog and run live, real-time, auction sales - the “Going once, Going twice” type.

- “Listen Bruce,” (that’s how I call him) “I’m on my way to working out. Did something crash?”
I don’t always think that the worst has happened, but you did just read the background.
- “No.”

I was expecting a long pause. In a way, I think he kind of enjoys those moments, where he knows I don’t know if it’s good or bad news. In a way, I think I actually do somehow enjoy them myself. But instead he said, “Are you next to a computer?”

- “No. I’m in the car. Should I turn back? What happened?”

I really hate to do this, but in order for his next sentence to make sense I have to go back exactly 95 years, to 1922 Tokyo, Japan.

Continue reading…

Posted by Brian Hicks on Wed, 09/06/2017 - 05:39

Breaking Down Decoders From the Bottom Up
Last week, we covered how to break down decoders by starting from the innermost, or topmost, part.
But what if you’re having trouble breaking things down from the top?
(Or you’re dealing with a really complex JSON schema?)

This week, let’s look at it from a different perspective: the outermost structure in (or the bottom up!)

Continue Reading

Posted by Brian Hicks on Wed, 08/30/2017 - 19:58

Breaking Out of Deeply Nested JSON Objects
A reader of the JSON Survival Kit wrote me with a question (lightly edited):

I’ve got a JSON string that works fine in JavaScript:

{
"Site1": {
"PC1": {
"ip": "x.x.x.x",
"version": "3"
},
"PC2": {
"ip": "x.x.x.x",
"version": "3"
}
},
"Site2": {
"PC1": {
"ip": "x.x.x.x",
"version": "3"
},
"PC2": {
"ip": "x.x.x.x",
"version": "3"
}
}
}

I really can’t figure out how to parse this–will your book help with nested JSON where the keys are different 2 or 3 levels deep?

If not, then I’ll just give up on Elm–as this is the first project that I’m trying to do, and something as basic as this, I’m finding impossible.

The biggest mindset shift you need to succeed with JSON Decoding is to think of your decoders like bricks.
(I’ve written about this before, and it’s chapter 1 of The JSON Survival Kit.)
You can combine bricks to build whatever you like; the same is true of decoders!

Continue Reading

Posted by Ilias Van Peer on Wed, 08/02/2017 - 13:09

The one runtime exception nearly every Elm developer will encounter sooner or later is this one, dealing with recursive JSON decoders:

Uncaught TypeError: Cannot read property ‘tag’ of undefined at runHelp

ContextLet’s say you are writing a decoder for a recursive structure:https://medium.com/media/df8ff3ab37ef34dbc9448b2e4de39a75/hrefWith hypothetical JSON looking like this:https://medium.com/media/4cc288b6dbf33b922d6961340a645ad7/hrefA first attemptThe most straightforward approach to this type of decoder, is to create a branch-decoder, a leaf-decoder and a tree-decoder, which would look something like this:https://medium.com/media/6b631a0e4873d6eae488b3a6238ad0e8/hrefHowever, Elm is an eager language, and functions are evaluated as soon as all of their arguments are passed. In the above example, where decoders are simply values, we’re dealing with recursively defined values, and you can’t do that in an eager language. Elm will, of course, point this out in its usual, friendly manner.https://medium.com/media/93d6928f2a64e31dea3288ad76f4b4d5/hrefIntroducing lazinessAfter reading the linked document, you know you need to introduce laziness using — in this case — Json.Decode.lazy. You may be wondering where to put the call to lazy: should you lazily refer from decoder to branchDecoder, or should it be the other way around?The slightly surprising answer is this:

There is no way to know for sure.

Elm orders its output based on how strongly connected different components are. In other words, a function that has more dependencies is more likely to appear later, a function that has fewer dependencies is more likely to appear earlier. In our case, that makes the most likely order leafDecoder -> branchDecoder -> decoder.If we lazily refer to branchDecoder from decoder, this order doesn't change; and branchDecoder will still eagerly refer to decoder whose definition appears only later in the compiled code.https://medium.com/media/6445861be604d4f845618466a9bf2c27/hrefLet’s have a look at what that would compile to.Sidenote: this is not quite what the compiled code looks like. I’ve stripped out the module-prefixes, partial application calls through the A* functions, and an Elm List really isn’t a JS array. However, it illustrates the core issue.https://medium.com/media/fec4f89b491d69bf97b3df0f63c82647/hrefNotice that there’s only a single function definition in this whole thing - everything else is eagerly evaluated and has all of its argument available. After all, decoders are values! Note, also, that branchDecoder is defined before decoder is defined; yet it references decoder. Since only function declarations are hoisted to the top in JavaScript; the above code can't actually work! decoder will be undefined when branchDecoder is used.A second attemptOur second option is moving the lazy call so branchDecoder lazily refers to decoder instead:https://medium.com/media/338c3e9ab3b910826e01b25f62f63d54/hrefAnother look at the pseudo-compiled code shows that we have reached our goal, this time:https://medium.com/media/3d7d32f4ca4c3b17457f938b6c2a5c22/hrefNow for the correct solutionThe order in which compiled results appear in the output isn’t something you, while writing Elm code, should worry about. Figuring out the strongly connected components to figure out where the lazy should go, is not what you want to be thinking about.Worse yet, slight changes to your decoders might result in the order changing in the compiled code!The only real option when dealing with this type of recursion is to introduce lazy in both places:https://medium.com/media/88a11f4ba8d1a760644b5e9177702c9d/hrefThis post was extracted from a document originally part of my Demystifying Decoders project. Learn more about that here:Demystifying Elm JSON DecodersHelp, my recursive Decoder caused a runtime exception! was originally published in Ilias Van Peer on Medium, where people are continuing the conversation by highlighting and responding to this story.

Posted by Brian Hicks on Thu, 07/27/2017 - 19:02

The State of Elm 2017 results are here!
I first presented these as two talks, one to Elm Europe and one to the Oslo Elm Day.
I’ve embedded the Elm Europe talk below (it’s shorter) but the longer version is on also YouTube.

Continue Reading

Posted by Ilias Van Peer on Sat, 07/08/2017 - 11:53

I figured it would be fun to take a tiny function and explain how it works line by line.https://medium.com/media/a2a9d0a1397e23b4532f2fb98cfe76ca/hrefLet’s examine that, line by line, function by function. Noting down the link to the documentation, the signature of each function used and what the inferred types look like at that point should prove — if nothing else — interesting to some!https://medium.com/media/9c2c32674172cb3125b68dad349bcadd/hrefDelayed HTTP requests in Elm, line-by-line was originally published in Ilias Van Peer on Medium, where people are continuing the conversation by highlighting and responding to this story.

Posted by Ilias Van Peer on Sat, 07/01/2017 - 19:38

elm-reactor is an underrated tool. Not only does it do on-demand recompilation of Elm source code, but it can serve up other assets, too.But did you know you can serve your own HTML with live-compiled Elm code, too? This is useful if you need JS interop or want to start your program with flags.The trick here is that elm-reactor exposes a “magical” /_compile directory — any elm file prefixed with that path will be pulled in and compiled on page-load.For example, start with a folder-structure like this:myProject/|- elm-package.json|- index.html`- src/ `- Main.elmPlacing your index.html at the same level as your elm-package.json means that running elm-reactor from your myProject folder will allow you to point your browser to http://localhost:8000/index.htmlAs for the contents of your index.html, start with something like this:<html><head> <style> /* custom styles? Sure! */ </style></head><body> <!-- Relative to index.html, main.elm lives in `src/Main.elm`. --> <!-- Prefixing that with `/_compile/` gives us magic! --> <script src="/_compile/src/Main.elm"></script> <script> var app = Elm.Main.fullscreen() // You could also pass flags, or setup some ports, ... </script></body></html>There, all set!Note that elm-reactor has also learned how to serve quite a few other file types with the correct content type headers, so pulling in some CSS, images or JSON should work, too.Shout-out to @ohanhi for the tip!Elm reactor and custom HTML was originally published in Ilias Van Peer on Medium, where people are continuing the conversation by highlighting and responding to this story.

Posted by Ilias Van Peer on Wed, 06/28/2017 - 16:27

JSON Decoders be what?Coming from JavaScript, where JSON is the most natural thing ever, having to write decoders to work with JSON in Elm is a mystifying experience. On some level, you understand that you need some way to convert these foreign objects into statically typed structures for safe use in Elm. And yet, it’s… weird.Some people learn best by reading about how they work, and looking at examples. Other people learn best by doing: writing decoders, from “very simple” to “I didn’t know you could do that”, progressively cranking up the complexity level, and knowing that what you wrote, is correct.To that end, I’ve put together a series of exercises that aim to walk you through writing JSON decoders. Each exercise aims to be a little more difficult than the one before, and introduce new concepts at a fairly reasonable pace.zwilias/elm-demystify-decodersThe code is there, all contributions are welcome, if you get stuck on something, create an issue and someone will help you out sooner or later.Now go and write some decoders!Demystifying Elm JSON Decoders was originally published in Ilias Van Peer on Medium, where people are continuing the conversation by highlighting and responding to this story.

Posted by Gizra on Tue, 05/23/2017 - 00:00

Chances are that you already using Travis or another Cool CI to execute your tests. Very often getting boolean or textual output from the execution is enough, because knowing which tests are failing is a good starting point to start to debug the problematic code. In our case, with WebdriverI/O (WDIO) and with an architecture where the frontend and backend are decoupled, it’s much more complicated.

It might be that the browser could not click on an element, or the frontend could not contact the backend, or the frontend has a runtime error (well, you might be faced with it, but at Gizra we use Elm, where it is practically impossible). Who knows, even the browser could crash due to lack of memory - the same applies to Travis too. One solution is to manually start reproducing what Travis does. It’s fun the first time, but doing it again and again is just a waste of time. But recently, our CTO, Amitai gave excellent pointers about dockerized Selenium and insisted that having video recordings is much better than simple static screenshots - and it was so true.

These days at Gizra - on client projects - we can benefit by knowing exactly how and why our browser-based tests failed. The fact that we already used Docker inside Travis helped a lot, but this additional video recording on the browser-based test makes the life of the developers much easier.

Ingredients

Let’s overview what’s bundled into Drupal Elm Starter, and who is responsible for what.

  • Upon a push, GitHub invokes Travis to start a build, that’s just the standard for many projects on GitHub for a long time.
  • Travis executes a set of shell scripts according to the build matrix. The only noteworthy thing is that using the build matrix with environment variables can be used to test the things in parallel - like one element of the matrix is the WDIO test, and another element could be any kind of Lint to scrutinize the code quality.
  • From this point, we only focus on one element of the build matrix. Docker Compose launches two containers, one with the application and the test code, the other with a Selenium Grid. It also helps the containers talk to each other via expressive hostnames.
  • The WDIO executes our test suites, but the Selenium host is not localhost, but rather the address of the other Docker container. This way Zalenium is able to record a video of the WDIO tests, it hosts the browser, the Selenium Grid and ffmpeg to encode the movie on-the-fly.
  • Google Drive hosts the videos of the failed tests. To use Google Drive programmatically, several steps are needed, but the gdrive uploader tool has excellent documentation.
  • In the very end, Gizra Robot posts a comment on the conversation thread of the pull request. Adding a robot user to GitHub is not different from adding a human - you can create a new GitHub user and dedicate it to this purpose. The exact process is documented in the repository.

The result

You can see an example video of the test on a recent pull request. The icing on the cake is that if you receive the GitHub notification email to your GMail inbox, you can launch a video straight from there via a YouTube player!

WebdriverI/O in action

Lessons learned

I joined Gizra three months ago, and the Gizra Way’s time-box/escalation system helped a lot to accomplish this task, where many layers of the CI stack were new to me. Needless to say, debugging Travis is hard. And then, you need to wait. And wait. A lot. Then your issue has a timebox on it, so hard things must be done quickly and by following best practices.

Seems impossible, right?

My experience is that this rigorous workflow helped me to find creative ways to solve the problems (not talking about ugly hacks here - just merely changing the way to find proper solutions), if the complexity is adequately calibrated to the developer, it triggers good stress that helps in problem solving too and contributes to the work satisfaction.

Let’s see how I was led to make it happen.

Dissect steps
It seems to be obvious that you need to break the problem into smaller chunks, but when the testability is so problematic, you must follow this principle very strictly. In this case, the most helpful was to test the different units in the simplest environment as possible. For instance there’s a Bash script that’s responsible for the GitHub upload. Instead of launching the script via Travis or via a similar local environment, in the native local environment, just feeding the script with the proper environment variables, what Travis would do, helped to speed up the process to almost real time debuggability.

Even a small Bash construct can be extracted and tested separately. Same for a curl invocation that posts a comment on GitHub. So in the end, I enjoyed the efficiency that came from the way of testing all the things with the minimally needed context - without all the hassle.

Invest in easy troubleshooting
It was a strong sign that we wanted to invest a significant amount to have this functionality at our project template, at Elm Starter, just to help future work. Similarly on the low level, it was mandatory at some point to be able to SSH into the Travis build. It’s enabled for private repositories, but in our case, it was mandatory to write to Travis support and this way, for our public repository, it was possible to use this functionality. It helped a lot to understand why the process behaves differently than at the local environment.

Contributing what you can
During the implementation, there were some issues with Zalenium, the side container, which provided Selenium Grid and the video recording (https://github.com/zalando/zalenium/pull/92). It got merged to upstream after 12 days, mostly the time the maintainer waited for my answer. It is just a little documentation fix, but it might save fellow developers frustration. On my side, I had the confirmation from the most capable person that I should not try to use --abort-on-exit with that container. Such scenarios reinforces the best practice, give back what you have, either it is knowledge, a patch or a full-blown solution.

Takeaway

The solution that is publicly available at the repository is easy to re-use in any project that has a similar browser-based test, the only criteria is that it should support the execution on a Selenium Grid. You might capture videos of your pure Simpletest, Behat, WDIO or Nightwatch.js (and who knows what kind of other test frameworks are out there in the wild) test suite and port this code from Drupal Elm Starter to easily understand why your test fails, the only criteria is that you should be able to execute Zalenium Docker container aside. Pull requests are more than welcome to make the process more robust or even sleeker!

Continue reading…

Posted by Brian Hicks on Mon, 05/01/2017 - 18:00

Sending Dates Through Elm Ports
So you’re sending dates through ports in your Elm application, and things are getting pretty confusing.
You send in a date, and you get…

Err "Expecting a String a _.date but instead got \"2017-05-01T12:45:00.000Z\""

Wait, what?
Isn’t that a string already?
What’s going wrong here?

Continue Reading

Posted by Brian Hicks on Mon, 05/01/2017 - 18:00

Sending Dates Through Elm Ports
So you’re sending dates through ports in your Elm application, and things are getting pretty confusing.
You send in a date, and you get…

Err "Expecting a String a _.date but instead got \"2017-05-01T12:45:00.000Z\""

Wait, what?
Isn’t that a string already?
What’s going wrong here?

Continue Reading

Posted by Ilias Van Peer on Thu, 04/27/2017 - 17:25

Or perhaps you did. Reminders can’t hurt, though.1. Changing types in record update syntaxRecord update syntax { a | b = c } is restricted in various ways: you cannot add or remove fields, but only change the values of fields. In that regard, they are very much like labeled tuples: the general shape can’t change. However, here’s something you can do that may come as a bit of a surprise: changing datatypes.Naturally, if you have a type alias that says some field needs to be a certain type, and you change that type somewhere, the resulting record will no longer conform to that type alias. Here are a few examples illustrating that.https://medium.com/media/0475bc2cca1916585141bdb86dd525a4/hrefWhen is this useful?As an example, imagine we use some browser detection library in JS like platform.js which can potentially result in null for every value. Let’s say we’re also taking a seed for a random number (because we don’t trust using the current time). We can do that as follows:https://medium.com/media/9461e6ecfd01b472d2028de6792e1ec5/href2. Ports and flags don’t require automatic conversionBoth ports and flags do automatic conversion of raw JS values to Elm types; but that comes with limitations, for example custom union types.Json.Encode.Value to the rescue! A Value type encapsulates a raw JS value; so annotating your ports or init function to accept a Value basically means that Elm won’t do its automatic conversion for those values. You will, however, need to handle the decoding and conversion yourself.The same goes for outgoing ports, where you can handle the encoding yourself before passing the value of to the port.Automatic conversion is sort of a relic from times when there wasn’t a streamlined Json.Decode library, and it was basically the only way to handle foreign values.When is this useful?

  • Decoupling your Elm types from your JS types
  • Handling union types
  • Control over what happened when you receive bad values — rather than an error, you get to graciously handle it in Elm.

3. Unsubscribe from outgoing portsLet’s say you have an outgoing port in your Elm app and subscribed to it from the JS side. How does one do that?You unsubscribe of course! Note that you need to pass the same callback to unsubscribe as you originally passed into subscribe — similar to working with event handlers. There’s even some magic so you can call unsubscribe from within your subscription handle, meaning you can pass this and unsubscribe without clutter!As illustration, have a look at the following Ellie. After 5 clicks, they will no longer be printed to the console.https://medium.com/media/493dfd0689f41341b713b936f0ea4fa8/href4. Pipelines are free!Or, more correctly — the right and left function application operators |> and <| are special cased in the compiler so they don’t cause any overhead.Thanks to Richard Feldman for this little gem.Naively, a |> b would compile into a function calling the |> operator with a and b as arguments; after which the |> operator would call b a. Under that assumption, simply writing it as b a instead would be more performant. However, that’s not actually what happens — the compiler knows about <| and |> and gives the special treatment; basically inlining their operation.This means that pipelines, which improve readability, don’t come at some sneaky cost; they’re free!5. Phantom types are a thingPhantom types are types in which type variables are declared on the right-hand side that aren’t used in the left-hand side. As an example type MyType a = My String where it’s clear that a isn’t used. This somewhat weird construct gives us the ability to express certain constrains about the values without directly affecting the values.Thanks to Martin Janiczek for brining this up in the Elm-lang slack earlier!For example, it becomes possible to create a FormData type that is Unsanitized by default, but allows sanitizing the input using a user-provided Sanitizer and restricts submiting to Sanitized data, like follows.https://medium.com/media/312b0f320740353f5b0438995166a140/hrefNow, on its own, this isn’t particularly useful — the same could be accomplished by putting the data in Unsanitized and Sanitized and throwing away the whole FormData type. However, this becomes useful when dealing with functions that don’t care about whether or not the data is sanitized, but just need a FormData a and can work on that type. In order to do that with two distinct types, you’d have to write different functions and give them different names, and double the maintenance effort.Googling around for phantom types will probably offer you a whole host of other use cases than this one, definitely worth checking out.So there you have it, some neat things you might not have known about Elm. If you enjoyed this post, feel free to check out my other posts and don’t hesitate to click that green little heart!5 Things you didn’t know about Elm was originally published in Ilias Van Peer on Medium, where people are continuing the conversation by highlighting and responding to this story.

Posted by Brian Hicks on Mon, 04/24/2017 - 18:00

Add Safety to Your Elm JSON Encoders With Fuzz Testing
How do you keep your JSON encoders, decoders, and model in sync?
You can skip fields in your encoder, right?
But should you?
And what about when you add new fields?
Decoders are a little easier, but you have to sync them up with your encoders or you’ll lose data.
And the worst part is that we can’t rely on the compiler to catch these classes of errors… argh!

This is a perfect situation for property tests (fuzz tests in elm-test lingo.)
The test system will keep us honest by giving us random values to test with.
You can assert that encoders and decoders mirror each other, and add a bit more safety to your app.

Continue Reading