Tuesday, April 30, 2013

Getting Started with a Browser Dart Test Suite

‹prev | My Chain | next›

One of the main reasons that I am switching the ICE Code Editor from JavaScript to Dart is for testing. I keep making all sorts of changes to a moderately large project with no safety net. With the Dart version, I will have type analysis safety net from the outset. Even so, I love me some tests.

The ICE Code Editor is very much a browser-based application. This means that the tests need a simple web page to serve as context for my tests. So I create the following as test/index.html:
<html>
<head>
  <title>ICE Test Suite</title>
  <script type="application/dart" src="editor_test.dart"></script>

  <script type="text/javascript">
    // start dart
    navigator.webkitStartDart();
  </script>
</head>

<body>
<h1>Test!</h1>

</body>
</html>
There is not much to this page. The body has an <h1>, which serves no purpose other than to have something on the page when viewed in a browser. The two <script> tags load the test suite and start the dart engine.

As for the test suite, I start very simple by testing some default values in the Editor class. The test outline will look something like this:
import 'package:unittest/unittest.dart';
import 'package:ice_code_editor/editor.dart';
import 'dart:html';

main() {
  group("defaults", () {
    // tests will go here...
  });
}
The import statements pull in the libraries that are needed for testing: the unit test library, the class being tested, and 'dart:html' for simple DOM manipulation and querying. Next comes the main() function. All Dart scripts need a main() entry point, so I oblige. I start by jamming all of my tests directly inside the body of the main() function. Eventually, I will pull them out, but this will do for now. Lastly, I add a test group(). This is not strictly necessary, but I seem to recall that certain test output formatters appreciate a group().

With that, I am ready to test the class. It is possible to disable auto-update of the preview layer when code changes take place. I am going to test that an Editor instance has auto-update on by default. The test for this is simple enough:
    test("defaults to auto-update the preview", () {
      var it = new Editor('ice');
      expect(it.autoupdate, equals(true));
    });
The entire content of editor_test.dart is now:
import 'package:unittest/unittest.dart';
import 'package:ice_code_editor/editor.dart';
import 'dart:html';

main() {
  group("defaults", () {
    test("defaults to auto-update the preview", () {
      var it = new Editor('ice');
      expect(it.autoupdate, equals(true));
    });
  });
}
To make that pass, I define the Editor class in lib/editor.dart as:
import 'dart:html';

class Editor {
  bool edit_only, autoupdate;
  String title;

  Editor(el, {this.edit_only, this.autoupdate:true, this.title}) {
  }
}
And it passes. When I load the test page up in Dartium and check the console, I see:



Unfortunately, the bug that suppresses console output in DumpRenderTree stills seems to be in place. When I try to test from the command-line, I see console output indicating that the unit test suite is starting, but nothing else:
➜  ice-code-editor git:(master) ✗ DumpRenderTree test/index.html      
CONSOLE MESSAGE: unittest-suite-wait-for-done
Content-Type: text/plain
layer at (0,0) size 800x600
  RenderView at (0,0) size 800x600
layer at (0,0) size 800x600
  RenderBlock {HTML} at (0,0) size 800x600
    RenderBody {BODY} at (8,8) size 784x571
      RenderBlock {H1} at (0,0) size 784x37
        RenderText {#text} at (0,0) size 69x36
          text run at (0,0) width 69: "Test!"
#EOF
#EOF
Happily, the submitter of that bug noted a workaround. I downgrade the unittest version on which I am depending by editing my package's pubspec.yaml to indicate that I want to peg ICE at version 4.0:
name: ice_code_editor
version: 0.0.1
description: Code Editor + Preview
author: Chris Strom <chris@eeecomputes.com>
homepage: https://github.com/eee-c/ice-code-editor
dependencies:
  unittest: 0.4.0
  js: any
Then re-run pub install:
➜  ice-code-editor git:(master) ✗ pub install
Resolving dependencies...
Dependencies installed!
I can then get the desired unit test output on the command-line as well:
➜  ice-code-editor git:(master) ✗ DumpRenderTree test/index.html
CONSOLE MESSAGE: unittest-suite-wait-for-done
Content-Type: text/plain
layer at (0,0) size 800x600
  RenderView at (0,0) size 800x600
layer at (0,0) size 800x600
  RenderBlock {HTML} at (0,0) size 800x600
    RenderBody {BODY} at (8,8) size 784x571
      RenderBlock {H1} at (0,0) size 784x37
        RenderText {#text} at (0,0) size 69x36
          text run at (0,0) width 69: "Test!"
#EOF
#EOF
CONSOLE MESSAGE: PASS: defaults defaults to auto-update the preview
CONSOLE MESSAGE: 
CONSOLE MESSAGE: All 1 tests passed.
CONSOLE MESSAGE: unittest-suite-success
For good measure, I ensure that dart_analyzer is happy:
➜  ice-code-editor git:(master) ✗ echo $?
0
Before calling it a night, I get started on the js-interop testing, which I need for things like incorporating ACE code editor. I will leave the actual testing until tomorrow, but I would like to ensure that turning it on does not break anything. So I add it to the Editor definition:
import 'dart:html';
import 'package:js/js.dart' as js;

class Editor {
  bool edit_only, autoupdate;
  String title;

  Editor(el, {this.edit_only:false, this.autoupdate:true, this.title}) {
    var context = js.context;
    context.ace.edit(el);
  }
}
The addition of the js import does not break anything, but trying to retrieve the js.context property cause all sort of bad:
unittest-suite-wait-for-done undefined:1
Uncaught ReferenceError: ReceivePortSync is not defined index.html:54
Exception: 'file:///home/chris/repos/ice-code-editor/test/packages/unittest/unittest.dart': Error: line 763 pos 28: type 'ExpectException' is not loaded
    String message = (e is ExpectException) ? e.message : 'Caught $e';
                           ^
malformed type used.
Stack Trace: #0      _registerException (file:///home/chris/repos/ice-code-editor/test/packages/unittest/unittest.dart:763:28)
#1      guardAsync (file:///home/chris/repos/ice-code-editor/test/packages/unittest/unittest.dart:744:23)
#2      _nextBatch._nextBatch (file:///home/chris/repos/ice-code-editor/test/packages/unittest/unittest.dart:782:23)
#3      runTests.runTests.<anonymous closure> (file:///home/chris/repos/ice-code-editor/test/packages/unittest/unittest.dart:731:16)
#4      _defer.<anonymous closure> (file:///home/chris/repos/ice-code-editor/test/packages/unittest/unittest.dart:688:13)
#5      _ReceivePortImpl._handleMessage (dart:isolate-patch:81:92)
This turns out to be due to the web page that provides context for the test suite. More specifically, it is not sufficient to start the Dart engine if I want to perform any kind of js-interop. There is additional setup required that, thankfully, is included in the browser dart package that is already part of my package dependencies. All I need do is replace this <script> tag in the web page:
<script type="text/javascript">
    // start dart
    navigator.webkitStartDart();
  </script>
With a single source <script> tag instead:
<script src="packages/browser/dart.js"></script>
Armed with that, I am ready to start testing the ACE setup via js-interop. And I'll pick back up there tomorrow.


Day #737

Monday, April 29, 2013

Initial Layout for the Dart Version of the ICE Code Editor

‹prev | My Chain | next›

With the big unknowns answered, I am ready to dive into converting the ICE Code Editor to Dart. The unknowns were the ability to interact with JavaScript libraries from Dart (js-interop does this with aplomb) and the ability to read existing localStorage data, which is compressed.

The Dart version of the ICE code editor will be a pub package. I have been messing about in that directory with some exploratory code, but I think now is the time to think about layout. Well, maybe not so much think about it as follow the layout documentation

So I make the sub-directories that I believe that I will need:
➜  ice-code-editor git:(master) ✗ mkdir docs lib packages test example
And create the metadata files:
➜  ice-code-editor git:(master) ✗ touch README.md LICENSE pubspec.yaml

While I am at it, I tell git to ignore the pubspec.lock file as well as the packages directory:
➜  ice-code-editor git:(master) ✗ cat <<-YAML > .gitignore
heredocd> packages
heredocd> pubspec.lock
heredocd> YAML
To sanity check my layout, I start work in the example sub-directory. I create a simple app.dart web server and a public sub-directory underneath that to hold an index.html web page:
<head>
  <script src="js/ace/ace.js" type="text/javascript" charset="utf-8"></script>
  <script src="packages/browser/dart.js"></script>
  <script type="application/dart">
    import 'package:ice_code_editor/ice.dart';
    main() => new ICE.Full('ace');
  </script>

</head>
<h1>Hello</h1>
<div style="width:600px; height: 400px" id="ace"></div>
I will create ice.dart back in the <PACKAGE_ROOT>/lib directory in a bit, but first I struggle with the location for the ACE code editor JavaScript files. I love Dart and all, but there is no way that I am going to attempt to reproduce ACE in Dart. Instead, I will continue to call the JavaScript code from Dart. But where to put the JavaScript code?

The pub documentation would seem to suggest that I put it in <PACKAGE_ROOT>/web, but I have no idea how to source that from a <script> tag if I follow that approach. I table that for now and include the JavaScript directly in <PACKAGE_ROOT>/example/public/js/ace. This is not a long term solution as anyone wanting to use the ICE code would also have to manually copy ACE into their application. Still, this ought to allow me to verify that everything else is in place.

Now, I can switch to <PACKAGE_ROOT>/lib to create the ICE.Full() constructor that is used in the example page. In ice.dart, I add:
import 'package:js/js.dart' as js;

class ICE {
  ICE.Full(el) {
    var context = js.context;
    context.ace.edit(el);
  }
}
With that, I can start the sample app up, load the homepage in Dartium and I see:



Nice! It just works. I have myself a decent start on the overall structure of my package. But where to put those JavaScript files?

The answer to that would seem to come from the browser package, which bundles the dart.js JavaScript file. If the Dart maintainers feel no compunction about including JavaScript directly in <PACKAGE_ROOT>/lib, then why should I? So I move the ace sub-directory under <PACKAGE_ROOT>/lib/ace. This ensures that ACE will be bundled with ICE (and that the ACE JavaScript API will be fixed in the package).

With that, I can modify the example page to point to the bundled ACE:
<head>
  <script src="packages/ice_code_editor/ace/ace.js" type="text/javascript" charset="utf-8"></script>
  <script src="packages/browser/dart.js"></script>
  <script type="application/dart">
    import 'package:ice_code_editor/ice.dart';
    main() => new ICE.Full('ace');
  </script>

</head>
<h1>Hello</h1>
<div style="width:600px; height: 400px" id="ace"></div>
And everything still works.

This seems like a good stopping point for tonight. I will pick back up with some tests tomorrow.


Day #736

Sunday, April 28, 2013

The Strange Tale of Dart, JavaScript, and Gzip Headers and Footers

‹prev | My Chain | next›

I continue my efforts to convert the ICE Code Editor from JavaScript to Dart. The two big unknowns before I started this were calling JavaScript libraries (e.g. ACE) from Dart and reading gzip data. It turns out that working with JavaScript in Dart is super easy, thanks to js-interop. Working with gzip compressed data in Dart is also easy. But I have trouble reading the data gzip'd with js-deflate

Jos Hirth pointed out that the Dart version was most likely doing what the gzip command-line version was doing: adding a standard gzip header and footer to the body of the deflated data. If that is the case, then I may have a decent migration strategy—add a few bytes before and after the old data and I ought to be good to go.

To test this theory, I start in JavaScript. I have the code that I want to deflate stored in code:
code = "<body></body>\n" +
  "<script src=\"http://gamingJS.com/Three.js\"></script>\n" +
  "<script src=\"http://gamingJS.com/ChromeFixes.js\"></script>\n" + 
  "<script>\n" +
  "  // Your code goes here...\n" +
  "</script>";
Next I use js-deflate to deflate this code into str_d:
str_d = RawDeflate.deflate(code)
This deflated string, str_d should serve as the body of the gzip data. Now I need the header and the footer. Per this onicos document, I should be able to make the header with:
header = [0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03].
  map(function(b) {return String.fromCharCode(b)}).
  join("")
Those are ten bytes that comprise the header, mapped into a string just as the deflated bytes were mapped into str_d. The first two bytes are always those values, per the documentation. The next, 0x08 signifies that the body contains deflate data. The remaining are supposed to hold Unix timestamp data, but I guess that this is not necessary. There are also one or two bytes that are supposed to hold optional data, but again, I leave them empty. The last byte could probably also be left empty, but I set it to 0x03 to signify Unix data.

As for the footer, it is supposed to hold 4 bytes of crc32 and 4 bytes describing the compressed size. For the time being, I leave them completely empty:
ooter = [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00].
  map(function(b) {String.fromCharCode(b)}).
  join("")
Hopefully this will result in a warning, but not an error. That is, hopefully, I can still gunzip this data even if a warning is given.

With that, I concatenate header, body, and footer into a single string and then convert from bytes to base64:
btoa(header + str_d + footer)
"H4sIAAAAAAAAA7NJyk+ptLPRB1NcNsXJRZkFJQrFRcm2ShklJQVW+vrpibmZeelewXrJ+bn6IRlFqal6WcVKQC0QtURocs4oys9NdcusSC3GrtWOS0FBX18hMr+0SCE5PyVVIT0/tVghI7UoVU9PjwuuHAAAAAAAAQAAAgAAAwAABAAABQAABgAABwA="
That looks promising—much closer to the Dart output from the other day which was:
H4sIAAAAAAAAA7JJyk+ptLPRB1NcNsXJRZkFJQrFRcm2ShklJQVW+vrpibmZeelewXrJ+bn6IRlFqal6WcVKQC0QtURocs4oys9NdcusSC3GrtWOS0FBX18hMr+0SCE5PyVVIT0/tVghI7UoVU9PjwuuHAAAAP//
To test the JavaScript result, I run it through the Linux base64 and gzip utilities:
➜  ice-code-editor git:(master) ✗ echo -n "H4sIAAAAAAAAA7NJyk+ptLPRB1NcNsXJRZkFJQrFRcm2ShklJQVW+vrpibmZeelewXrJ+bn6IRlFqal6WcVKQC0QtURocs4oys9NdcusSC3GrtWOS0FBX18hMr+0SCE5PyVVIT0/tVghI7UoVU9PjwuuHAAAAAAAAQAAAgAAAwAABAAABQAABgAABwA=" | base64 -d | gunzip -dc
<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>
gzip: stdin: invalid compressed data--crc error

gzip: stdin: invalid compressed data--length error
Success! As feared, I see crc32 and length errors, but nonetheless I am finally able to gunzip the JavaScript data.

Now that I understand everything, let's see if I can solve this in Dart. That is, can I take the body that was deflated in JavaScript and inflate it in Dart? The base64 and gzip'd version of the code is stored in the ICE Code Editor's localStorage as:
"s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA=="
The built-in Zlib library in Dart operates on streams. So I take this base64/deflated data, add it to a stream, pass the stream through an instance of ZlibInflater, and then finally fold and print the result:
import 'dart:async';
import 'dart:io';
import 'dart:crypto';

main() {
  var data = "s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==";

  var controller = new StreamController();
  controller.stream
    .transform(new ZLibInflater())
    .fold([], (buffer, data) {
      buffer.addAll(data);
      return buffer;
    })
    .then((inflated) {
      print(new String.fromCharCodes(inflated));
    });
  controller.add(CryptoUtils.base64StringToBytes(data));
  controller.close();
}
This fails when I run it because the ZLibInflater expects a header and a footer:
➜  ice-code-editor git:(master) ✗ dart test.dart
Uncaught Error: InternalError: 'Filter error, bad data'
Unhandled exception:
InternalError: 'Filter error, bad data'
So, add the 10 byte header to the stream before adding the body:
var controller = new StreamController();
  controller.stream
    .transform(new ZLibInflater())
    .fold([], (buffer, data) {
      buffer.addAll(data);
      return buffer;
    })
    .then((inflated) {
      print(new String.fromCharCodes(inflated));
    });
  controller.add([0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03]);
  controller.add(CryptoUtils.base64StringToBytes(data));
  controller.close();
}
Which results in:
➜  ice-code-editor git:(master) ✗ dart test.dart
<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>
Huzzah! I finally have it. Given js-deflated & base64 encoded data that is stored in the ICE Editor's localStorage, I can read it back in Dart. Interestingly, I do not even need the footer. In fact, if I add the footer to the stream, I again get bad filter data—no doubt due to the bogus crc32 and length information that I feed it. No matter, the footer is not necessary to get what I need.

Of course, none of this really matters until I can convince the fine Dart folks that the Zlib libraries belong in "dart:cypto" instead of "dart:io". The former is available to browsers, which I where I need it if I am to read (and ultimately write) localStorage. The latter is only available server-side which is of no use to me. Thankfully, I can go on merrily using Dart's js-interop to use js-deflate for the time being. And hopefully the time being won't be too long.


Day #735

Saturday, April 27, 2013

js-deflate in Dart

‹prev | My Chain | next›

I love me a good mystery.

Last night, I figured out how to gzip and base64 encode data in Dart. That was not the mystery. It worked so well, that I could even pass the result to the linux base64 and gzip utilities. The mystery is why the JavaScript equivalents, js-deflate and btoa, do not produce the same result.

Since Dart seems compatible with the linux utilities, it would seem that it is "right". So my first inclination is to see if I can get the JavaScript version right. I start with the same code string as yesterday:
> str
"<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>"
Instead of deflating / zipping that string immediately, I try base64 encoding, then deflating it, and then base64 encoding it again:
> btoa(RawDeflate.deflate(btoa(str)))
"Rc9Bb4MgGIDhH9QLQU3k0MO0mcAoYzrb+t0UFyvD4RLF4a9fdmh2e05v8qqCe6CfB5WTscNlogaXaixDe8sQozI0OxpbWiJ9cl7g/t5d7QpofheT9F1F6pZyC1XyrQP7a+xNxGdN3w65+Xl40BE3quKup+X2OqYe8PPSXpOvyp7XBpNFIOn0RBaoYf4oLruY5p2ZOBWRNHqyW3+KXxT9N8ufvAgZdNElsEJ6KOoBMLE6ZA5u3IoxXnOz+ceHGo7HXw=="
I then take the result, pass it to the linux base64 and gzip utilities to find:
➜  ice-code-editor git:(master) ✗ echo -n "Rc9Bb4MgGIDhH9QLQU3k0MO0mcAoYzrb+t0UFyvD4RLF4a9fdmh2e05v8qqCe6CfB5WTscNlogaXaixDe8sQozI0OxpbWiJ9cl7g/t5d7QpofheT9F1F6pZyC1XyrQP7a+xNxGdN3w65+Xl40BE3quKup+X2OqYe8PPSXpOvyp7XBpNFIOn0RBaoYf4oLruY5p2ZOBWRNHqyW3+KXxT9N8ufvAgZdNElsEJ6KOoBMLE6ZA5u3IoxXnOz+ceHGo7HXw==" | base64 -d | gunzip -dc

gzip: stdin: not in gzip format
Blah.

So it really seems like the js-deflate is not producing gzip compatible data.

So let's see what Dart can do with the JavaScript deflate implementation:
<head>
  <script src="scripts/rawdeflate.js" type="text/javascript" charset="utf-8"></script>
  <script src="scripts/ace/ace.js" type="text/javascript" charset="utf-8"></script>
  <script type="application/dart" src="scripts/ice.dart"></script>
  <script src="scripts/packages/browser/dart.js"></script>
</head>
<h1>Hello</h1>
<div style="width:600px; height: 400px" id="ace"></div>
Next, I modify my ice.dart script to deflate the same code, this time using the JavaScipt js-deflate implementation, and then base64 encode it:
import 'dart:html';
import 'dart:async';
import 'dart:crypto';
import 'package:js/js.dart' as js;

main() {
  var context = js.context;
  // ...
  var code = '''<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>''';

  var str_d = context.RawDeflate.deflate(code);
  print(str_d);
  print(CryptoUtils.bytesToBase64(str_d.codeUnits));
}
The result in the Dartium console:
s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==
By way of comparison, the pure JavaScript result is:
s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==
So in then end, I have no idea what the JavaScript function is doing. What I do know is that the JavaScript btoa() and Dart bytesToBase64() produce identical results. I also know that I can deflate identically in both Dart and JavaScript thanks to Dart's js-interop.

Sadly, the mystery remains. But happily for my efforts to convert the ICE Code Editor to Dart, I can still invoke the screwy JavaScript implementation and continue merrily on my way. It comes at the expense of being required to load in yet another JavaScript library, but only until I come up with a migration scheme.


Day #734

Friday, April 26, 2013

Gzip and Base64 in Dart

‹prev | My Chain | next›

To convert the ICE Code Editor to Dart, I will need the ability to read and write base64, gzip'd strings. The ICE Code editor gzips and base64 encodes code before storing it to localStorage. Unfortunately, this is almost certainly not going to work in Dart (at least not currently) because Dart's ZLibDeflater and ZLibInflater are part of the "dart:io" package, which is not available in the browser (noooooooooooo!!!!). Even so, I would like to give it a go just to see if it is capable of working with the equivalent JavaScript data.

The JavaScript files use js-deflate for the gzip compression/decompression and the built-in window.btoa() and window.atob() methods for base64 encoding. The result is that a simple code sample like:
<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>
Is stored in ICE as:
s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==
My first thought it to try to decode that in Dart. I import all of the necessary packages, declare the encoded code as str and then build up streams:
import 'dart:async';
import 'dart:io';
import 'dart:crypto';

main() {
  var str = "s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==";

  var controller = new StreamController();
  controller.stream
    .transform(new ZLibInflater())
    .fold([], (buffer, data) {
      print(data);
      buffer.addAll(data);
      return buffer;
    })
    .then((inflated) {
      print(new String.fromCharCodes(inflated));
      print(inflated);
    });
  controller.add(CryptoUtils.base64StringToBytes(str));
  controller.close();
}
I create a vanilla stream controller to which I can add data. The data that I add is the base64 decoded version of str. In the stream, I transform the data with a ZlibInflater instance, fold all of the data into a single buffer and then print out the resulting string. I think that ought to work, but...
➜  ice-code-editor git:(master) ✗ dart test.dart
s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==
[179, 73, 202, 79, 169, 180, 179, 209, 7, 83, 92, 54, 197, 201, 69, 153, 5, 37, 10, 197, 69, 201, 182, 74, 25, 37, 37, 5, 86, 250, 250, 233, 137, 185, 153, 121, 233, 94, 193, 122, 201, 249, 185, 250, 33, 25, 69, 169, 169, 122, 89, 197, 74, 64, 45, 16, 181, 68, 104, 114, 206, 40, 202, 207, 77, 117, 203, 172, 72, 45, 198, 174, 213, 142, 75, 65, 65, 95, 95, 33, 50, 191, 180, 72, 33, 57, 63, 37, 85, 33, 61, 63, 181, 88, 33, 35, 181, 40, 85, 79, 79, 143, 11, 174, 28, 0]
Uncaught Error: InternalError: 'Filter error, bad data'
Unhandled exception:
InternalError: 'Filter error, bad data'
#0      _FutureImpl._scheduleUnhandledError.<anonymous closure> (dart:async:325:9)
#1      Timer.run.<anonymous closure> (dart:async:2251:21)
#2      Timer.run.<anonymous closure> (dart:async:2259:13)
#3      Timer.Timer.<anonymous closure> (dart:async-patch:15:15)
#4      _Timer._createTimerHandler._handleTimeout (dart:io:6697:28)
#5      _Timer._createTimerHandler._handleTimeout (dart:io:6705:7)
#6      _Timer._createTimerHandler.<anonymous closure> (dart:io:6713:23)
#7      _ReceivePortImpl._handleMessage (dart:isolate-patch:81:92)
Ick.

I take an alternate approach to try to compress the same string to see what data comes back:
import 'dart:async';
import 'dart:io';
import 'dart:crypto';

main() {
  var str = "s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==";

  var code = '''<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>''';

  var controller = new StreamController();
  controller.stream
    .transform(new ZLibDeflater())
    .fold([], (buffer, data) {
      print(data);
      buffer.addAll(data);
      return buffer;
    })
    .then((inflated) {
      print(CryptoUtils.bytesToBase64(inflated));
      print(inflated);
    });
  controller.add(code.codeUnits);
  controller.close();
}
This results in a base64 string of:
H4sIAAAAAAAAA7JJyk+ptLPRB1NcNsXJRZkFJQrFRcm2ShklJQVW+vrpibmZeelewXrJ+bn6IRlFqal6WcVKQC0QtURocs4oys9NdcusSC3GrtWOS0FBX18hMr+0SCE5PyVVIT0/tVghI7UoVU9PjwuuHAAAAP//
Which is larger than what the JavaScript version has created:
s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==
I am at a loss to explain the difference, but the Dart version seems legit—I can even use this on the command line:
➜  ice-code-editor git:(master) ✗ echo -n "H4sIAAAAAAAAA7JJyk+ptLPRB1NcNsXJRZkFJQrFRcm2ShklJQVW+vrpibmZeelewXrJ+bn6IRlFqal6WcVKQC0QtURocs4oys9NdcusSC3GrtWOS0FBX18hMr+0SCE5PyVVIT0/tVghI7UoVU9PjwuuHAAAAP//" | base64 -d | gunzip -dc
<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>
gzip: stdin: unexpected end of file
While the JavaScript version will not work:
➜  ice-code-editor git:(master) ✗ echo -n "s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==" | base64 -d | gunzip -dc

gzip: stdin: not in gzip format
It seems unlikely that I will be able to figure out why the two produce such different results. Sadly, it hardly matters. Since the Dart zlib classes are only available in "dart:io", which is only in the server-side VM, not the browser VM, I cannot use it anyway to store and retrieve data from localStorage. In the end, I will likely use js-interop to continue using the JavaScript version.


Day #733

Thursday, April 25, 2013

Using the ACE Code Editor with Dart

‹prev | My Chain | next›

Having built the ICE Code Editor out into separate, reusable classes, I have learned much. Mostly I have learned that I really do not enjoy building large applications in JavaScript. I have to compile it constantly into a single downloadable file. While developing, I have to be sure to load all of the individual files in the right order. I have to be picky about the libraries that I pull in. Coding OO JavaScript is just ugly. And forget about testing. So I wonder, how easy would it be to do this in Dart?

Part of the trouble with Dart is that I really do not want to reimplement the actual editor part of ICE:



That comes from the ACE project and it way more work than I want to get into right now. But then again, there is the js-interop package—maybe that will let me interact with ACE, while still enjoying the benefits of Dart.

In the scripts sub-directory, I create a pubspec.yaml:
name: Testing ICE, ICE, Dart
dependencies:
  unittest: any
  js: any
Then I can pub install to get:
➜  scripts git:(master) ✗ pub install
Resolving dependencies...
Downloading meta 0.5.0+1...
Downloading browser 0.5.0+1...
Downloading unittest 0.5.0+1...
Downloading js 0.0.22...
Dependencies installed!
I start with a simple web page:
<head>
  <script type="application/dart" src="scripts/ice.dart"></script>
  <script>
  navigator.webkitStartDart();
  </script>
</head>
<h1>Hello</h1>
Unfortunately, even with a very simple ice.dart script file:
import 'dart:html';
import 'package:js/js.dart' as js;

main() {
  var context = js.context;
}
I run into a slew of problems in the console of Dartium:
Uncaught ReferenceError: ReceivePortSync is not defined localhost/:54
Exception: The null object does not have a method 'callSync'.

NoSuchMethodError : method not found: 'callSync'
Receiver: null
Arguments: [GrowableObjectArray]
Stack Trace: #0      Object.noSuchMethod (dart:core-patch:1907:25)
#1      _enterScope (http://localhost:8000/scripts/packages/js/js.dart:749:35)
#2      _enterScopeIfNeeded (http://localhost:8000/scripts/packages/js/js.dart:728:28)
#3      context (http://localhost:8000/scripts/packages/js/js.dart:717:22)
#4      main (http://localhost:8000/scripts/ice.dart:5:20)
 
It takes me a bit to realize that I cannot use my simple navigator.webkitStartDart(). Instead, I should rely on one of the dependencies that was installed to do this for me. So I remove that line and source dart.js from the browser package:
<head>
  <script src="scripts/ace/ace.js" type="text/javascript" charset="utf-8"></script>
  <script type="application/dart" src="scripts/ice.dart"></script>
  <script src="scripts/packages/browser/dart.js"></script>
</head>
<h1>Hello</h1>
With that, I am ready to roll.

So, back in ice.dart I tell the JavaScript context to start ACE:
import 'dart:html';
import 'package:js/js.dart' as js;

main() {
  var context = js.context;

  context.ace.edit('ace');
}
And it works!



Well, not entirely. The actual editor does not show up, but the ACE DOM elements are very much there. Satisfied that this seems doable, I call it a night here. I will pick back up tomorrow with actually showing the editor and maybe even doing ICE-like stuff with it.

Update figured it out. I had a typo in the style attribute for the ACE <div>. This fixes the problem:
<head>
  <script src="scripts/ace/ace.js" type="text/javascript" charset="utf-8"></script>
  <script type="application/dart" src="scripts/ice.dart"></script>
  <script src="scripts/packages/browser/dart.js"></script>
</head>
<h1>Hello</h1>
<div style="width:600px; height: 400px" id="ace"></div>
With that, I have Dart pulling ACE into Dartium:




Day #732

Wednesday, April 24, 2013

Show Preview Only in Viewport

‹prev | My Chain | next›

There are times that I will want several ICE Code Editor instances embedded on the same page. I definitely do not want each of the instances running its visualization at the same time—that's a good way to bring some machines to their proverbial knees. What would be ideal would be for the visualization to run only when in the viewport.

To decide if an element is in the viewport, I borrow from a StackOverflow solution and use:
function isElementInViewport(el) {
  var rect = el.getBoundingClientRect();

  return (
    rect.top >= 0 &&
    rect.left >= 0 &&
    rect.bottom <= window.innerHeight &&
    rect.right <= window.innerWidth
  );
}
Since the ICE Code Editor is intended to demonstrate Three.js visualizations (and WebGL visualizations in particular), there is no need to support older browsers or any version of IE. In other words, I can make use of the getBoundlingClientRect() method without worry.

Armed with that, I can build an addEventListeners() method for the ICE.Embedded class:
Embedded.prototype.addEventListeners = function() {
  var that = this;
  function showPreview() { /* ... */ }

  function hidePreview() { /* ... */ }

  document.addEventListener('scroll', showPreview);
  document.addEventListener('load', hidePreview);
  document.addEventListener('scroll', hidePreview);
};
I am not sold on the names for showPreview() and hidePreview(). Then again, these are internal to the method so the context is set. Besides, both sound better than show-preview-if-entirely-in-viewport.

I express the showPreview() function as:
  function showPreview() {
    if (!isElementInViewport(that.editor.preview_el)) return;
    if (!that.editor.isPreviewHidden()) return;
    that.editor.updatePreview();
  }
In other words, do not show if the preview is not in the view port; do not show if the preview is already showing; otherwise go ahead and show it.

The hidePreview() function is similar:
  function hidePreview() {
    if (isElementInViewport(that.editor.preview_el)) return;
    setTimeout(function(){that.editor.hidePreview();}, 100);
  }
I find that the timeout in there is necessary to account for load times of the previewed code's scripts.

With that, I have no preview when the editor is partially outside of the viewport:



But scrolling the editor into view fires up the preview:



I may eventually switch so that the editor only needs to be 80% of the way into the screen, but this is a reasonable first take.


Day #731

Tuesday, April 23, 2013

Tracer Bullets, Chaining Iterators, and Dynamic RegExp Fun

‹prev | My Chain | next›

Last night was one of those posts with a frantic race to the end. I got it working, but knew full well that there were still a few problems. The “it” in question is the navigation from embedded to full screen ICE Code Editor, which is initiated by clicking the SVG expand icon in the embedded editor:



In addition to transferring code between the two versions of the editor, I am also passing title information. Therein lies the problem. The title is transferred, but I am making no allowances for an existing project with the same name. The end result is that I have multiple projects with the same name:



Worse is that, if I try to delete one, both end up being removed.

As an initial workaround, I will add the copy number in parentheses after the project name. That is, if “Shapes Demo” already exists, then expanding another one should result in “Shapes Demo (1)”. To get there, I will engage in some intermediate range tracer bullets. Normally tracer bullets help build up nearby methods. In this case, I need to transfer information from the embedded ICE to the large ICE, determine if a project of the same name exists and then I'll try to illuminate my target. I could try to break this down into smaller bits, but the transfer of information complicates things. Anyhow...

The ICE.Full class does a bunch of work on the URL has, then end result being a call to the create() method of ICE.Store:
function processLocationHash() {
  // ...
  store.create(ICE.decode(hash.substr(2)), title);
}
So my tracer bullets will go one step further. In ICE.Store, I add a hasProjectNamed() method and put it to use in create():
Store.prototype.create = function(code, title) {
  if (!title) title = this._nextUntitled();
  if (this.hasProjectNamed(title)) title = title + ' 01';
  // ...
};

Store.prototype.hasProjectNamed = function(title) {
  return this.documents.some(function(doc) {
      return doc.filename == title;
    });
};
The ICE Code Editor will only work with modern browsers, so I have no compunction in using the some() iterator method on my document store. With that some() call, if any of the documents in localStorage have the same file as the new title, then the store does have a project named the same thing, and hasProjectNamed() returns true.

If my intermediate range tracer bullets are working, then I ought to have a project named “Shapes Demo 01” after navigating from the embedded ICE to the full screen version. And, indeed I do:



Now that I know that I am pointing in the right direction, it is time to hit the actual target. I replace the tracer bullet next title with a call to a new method, _nextProjectNamed():
Store.prototype.create = function(code, title) {
  if (!title) title = 'Untitled';
  if (this.hasProjectNamed(title)) title = this._nextProjectNamed(title);
  // ...
};
The _nextProjectNamed() method is some good old iterator-chaining, dynamic-regexp fun:
Store.prototype._nextProjectNamed = function(title) {
  title.replace(/\s*\(\d+\)$/, '');

  var nums = this.documents.
    filter(function(doc) {
      return doc.filename == title ||
             doc.filename.match(new RegExp('^' + title + ' \\(\\d+\\)$'));
    }).
    map(function(doc) {
      var num = doc.filename.
        replace(new RegExp('^' + title + '\\s*'), '').
        replace(/[)(\\s]+/g, '');
      return parseInt(num, 10);
    }).
    filter(function (num) {
      return !isNaN(num);
    }).
    sort();

  var next = nums.length ? nums[nums.length-1] + 1 : 1;
  return title + ' (' + next + ')';
};
As hairy as that looks, it is mostly based on prior work. It filters the entire localStorage list of documents for the ones that match the supplied title, then it strips everything but the copy number (if present) from the stored file names, then it ignores anything that is not a number from that list. The end result is a sorted list of copy numbers that can be used to build the next copy number.

That does the trick, even if intermediate copies are deleted:



I feel much better about this approach. I will likely take some time tomorrow to poke around the edges, but hopefully I am nearing the end of this batch of changes to ICE.


Day #730

Monday, April 22, 2013

Passing Data with Location Hash

‹prev | My Chain | next›

Now that I have a pretty SVG icon for expanding the embedded version of the ICE Code Editor, it is time to actually make it do something.



I think the easiest thing for now is to send the sourcecode, and a title, to the full-screen editor at http://gamingjs.com/ice. I can build from that later. The sourcecode should be fairly straight-forward. The ICE Code Editor already boasts the ability to gzip the code for storage and sharing. Something is sure to go wrong in practice, but in theory I ought to be able to make it work.

As for the title, I think this is probably something else to include in the <script> tag that contains the code on the page:
<script type="text/ice-code" id="code-001" line="20">
-script src="http://gamingJS.com/Three.js"
-script src="http://gamingJS.com/ChromeFixes.js"
// This is where stuff in our game will happen:
var scene = new THREE.Scene();
// ...
</script>
The -script lines are my clever way of sourcing external JavaScript libraries in the sample code without using <script> tags that would otherwise close the outer <script> tag. The outer <script> tag is already parsed by the ICE.Embedded class, so telling it to look for a title attribute is no trouble:
Embedded.prototype.attributesFromSource = function() {
  // ...
  if (this.script.attributes.title) {
    this.title = this.script.attributes.title.value;
  }
};
This should extract a title from the <script> tag of the form:
<script type="text/ice-code" title="Shapes Demo" line="20">
-script src="http://gamingJS.com/Three.js"
-script src="http://gamingJS.com/ChromeFixes.js"
  // This is where stuff in our game will happen:
  var scene = new THREE.Scene();
  // ...
</script>
I can then use this in the title supplied to the full-screen version of the ICE Code Editor:
Embedded.prototype.addControls = function() {
  var el = document.createElement('div');
  el.className = 'icon';
  document.body.appendChild(el);

  var that = this;
  el.addEventListener('click', function() {
    window.location = 'http://localhost:3000/#B/' +
      ICE.encode(that.editor.getContent()) +
      '?title=' + encodeURIComponent(that.title);
  });
  // ...
};
The full screen version of ICE is attached with the attachFull() method:
<script>
  ICE.attachFull();
</script>
I had not pulled in the URL parsing code, so I do that now:
function attachFull() {
  var el = createElements();
  setFlags();
  store = new ICE.Store();
  editor = new ICE.Editor(el, {edit_only: EDIT_ONLY});
  applyStyles();
  readLocationHash();
  editor.setContent(store.current.code);
  // ...
}
Most of that method sorts out whether or not code is embedded in the hashtag of the URL. If it is, then I need to parse it and read the title from the query string:
function readLocationHash() {
  var title;
  if (window.location.search.indexOf('title=')) {
    title = window.location.search.substr(window.location.search.indexOf('title=') + 6);
    title = decodeURIComponent(title);
  }
  store.create(ICE.decode(hash.substr(2)), title);
  window.location.hash = '';
  window.location.search ='';
}
I also have to remember to call the built-in decodeURIComponent. Happily, the ICE.Store class does the rest. I supply the decoded source code and title and it is stored in localStorage for immediate use.



That does the trick. After triggering the click event on the embedded code, I create a new entry in the localStorage that I can immediately start playing with. I still need to account for the case in which the title already exists, but this seems a decent stopping point for the night.


Day #729

Sunday, April 21, 2013

Animating SVG with JavaScript

‹prev | My Chain | next›

I donated to the Batch Icon Set used in this post. You should too!

I was able to add and SVG icon to the ICE Code Editor yesterday:



Today I would to add simple animation to it. Rather than start with the icon being large and dark, I would prefer that it be a lighter grey and a bit smaller. When the reader moves the mouse pointer over it, then it should grow to a larger size with a darker coloring.

My initial thought is to use some CSS3 to animate the icon on :hover (and when the mouse is no longer hovering over the icon). I cannot seem to get that working when I try applying the styles directly to the SVG. So instead, I insert a containing <div> tag to which I can apply CSS3 styles and which can hold the SVG element:
Embedded.prototype.addControls = function() {
  var el = document.createElement('div');
  document.body.appendChild(el);

  el.style.position = 'absolute';
  el.style.bottom = '20px';
  el.style.right = '40px';
  el.style.cursor = 'pointer';
  el.style.width = '25px';
  el.style.height = '25px';
  this.editor.editor_el.appendChild(el);
  // ...
};
This seems to work, except that I am not going to get my nice CSS3 animations on-hover by placing styles directly on that element. So I remove the explicit style property settings and instead add manual CSS to the bottom of the method (borrow from a StackOverflow solution):
Embedded.prototype.addControls = function() {
  var el = document.createElement('div');
  el.className = 'icon';
  document.body.appendChild(el);
  this.editor.editor_el.appendChild(el);
  // ...
  var head = document.getElementsByTagName('head')[0];
  var style = document.createElement('style');
  style.type = 'text/css';

  var declarations = document.createTextNode(
    '.icon {' +
      'position: absolute; ' +
      'cursor: pointer; ' +
      'fill: rgba(204, 204, 204, 0.8); ' +
      'width: 25px; ' +
      'height: 25px; ' +
      'bottom: 10px; ' +
      'right: 30px; ' +
      'transition: all 0.2s ease-in-out; ' +
    '} ' +
    '.icon:hover {' +
      'fill: rgba(0, 0, 0, 0.8); ' +
      'width: 32px; ' +
      'height: 32px; ' +
      'bottom: 6px; ' +
      'right: 26px; ' +
    '}'
  );

  style.appendChild(declarations);
  head.appendChild(style);
};
In ordinary web pages, I might add this as a separate CSS stylesheet. Since I am building an embeddable widget, I opt to build the CSS in JavaScript—position, color, size and all.

The most important aspect of this is the transition property. I am using all so that all CSS properties will transition at the same speed when switching from .icon to .icon:hover. Also of note is the use of the fill property, which tells the SVG element what color to use when drawing itself. I change the bottom and right absolute positioning of the containing <div> on hover to give the appearance that the animation originates from the center of the icon.

The entire method, container element, SVG element, and CSS is then:
Embedded.prototype.addControls = function() {
  var el = document.createElement('div');
  el.className = 'icon';
  document.body.appendChild(el);

  this.editor.editor_el.appendChild(el);

  var svg = document.createElementNS('http://www.w3.org/2000/svg', 'svg');
  svg.setAttribute('viewBox', '0 0 48 48');

  var path = document.createElementNS('http://www.w3.org/2000/svg', 'path');
  path.setAttribute('d', 'M 11.742,7.50 L 20.121,15.879 C 21.294,17.052 21.294,18.948 20.121,20.121 C 18.948,21.294 17.052,21.294 15.879,20.121 L 7.50,11.742 L 5.121,14.121 C 4.548,14.694 3.78,15.00 3.00,15.00 C 2.613,15.00 2.223,14.925 1.851,14.772 C 0.732,14.307 0.00,13.212 0.00,12.00 L 0.00,3.00 C 0.00,1.344 1.344,0.00 3.00,0.00 L 12.00,0.00 C 13.212,0.00 14.307,0.732 14.772,1.851 C 15.237,2.973 14.979,4.263 14.121,5.121 L 11.742,7.50 ZM 45.00,0.00 C 46.659,0.00 48.00,1.344 48.00,3.00 L 48.00,12.00 C 48.00,13.212 47.268,14.307 46.149,14.772 C 45.777,14.925 45.387,15.00 45.00,15.00 C 44.22,15.00 43.452,14.694 42.879,14.121 L 40.50,11.742 L 32.121,20.121 C 30.948,21.294 29.052,21.294 27.879,20.121 C 26.706,18.948 26.706,17.052 27.879,15.879 L 36.258,7.50 L 33.879,5.121 C 33.021,4.263 32.766,2.973 33.228,1.851 C 33.69,0.732 34.788,0.00 36.00,0.00 L 45.00,0.00 ZM 15.879,27.879 C 17.052,26.706 18.948,26.706 20.121,27.879 C 21.294,29.052 21.294,30.948 20.121,32.121 L 11.742,40.50 L 14.121,42.879 C 14.979,43.737 15.237,45.027 14.772,46.149 C 14.307,47.268 13.212,48.00 12.00,48.00 L 3.00,48.00 C 1.344,48.00 0.00,46.659 0.00,45.00 L 0.00,36.00 C 0.00,34.788 0.732,33.69 1.851,33.228 C 2.223,33.075 2.613,33.00 3.00,33.00 C 3.78,33.00 4.548,33.306 5.121,33.879 L 7.50,36.258 L 15.879,27.879 ZM 46.149,33.228 C 47.268,33.69 48.00,34.788 48.00,36.00 L 48.00,45.00 C 48.00,46.659 46.659,48.00 45.00,48.00 L 36.00,48.00 C 34.788,48.00 33.69,47.268 33.228,46.149 C 32.766,45.027 33.021,43.737 33.879,42.879 L 36.258,40.50 L 27.879,32.121 C 26.706,30.948 26.706,29.052 27.879,27.879 C 29.052,26.706 30.948,26.706 32.121,27.879 L 40.50,36.258 L 42.879,33.879 C 43.452,33.306 44.22,33.00 45.00,33.00 C 45.387,33.00 45.777,33.075 46.149,33.228 Z');

  svg.appendChild(path);

  el.appendChild(svg);

  var head = document.getElementsByTagName('head')[0];
  var style = document.createElement('style');
  style.type = 'text/css';

  var declarations = document.createTextNode(
    '.icon {' +
      'position: absolute; ' +
      'cursor: pointer; ' +
      'fill: rgba(204, 204, 204, 0.8); ' +
      'width: 25px; ' +
      'height: 25px; ' +
      'bottom: 10px; ' +
      'right: 30px; ' +
      'transition: all 0.2s ease-in-out; ' +
    '} ' +
    '.icon:hover {' +
      'fill: rgba(0, 0, 0, 0.8); ' +
      'width: 32px; ' +
      'height: 32px; ' +
      'bottom: 6px; ' +
      'right: 26px; ' +
    '}'
  );

  style.appendChild(declarations);
  head.appendChild(style);
};
And the effect is quite pleasant. It looks nice when small:



And it looks nice when the pointer is hovering over the element:



Best of all, it works in both Firefox and Chrome (IE's inability to perform WebGL makes it a non-starter for this).

I had thought to explore SVG's built-in animation as well, but this approach seems quite solid.

Day #728

Saturday, April 20, 2013

SVG Icons

‹prev | My Chain | next›

I am a sucker for icons. I really appreciate the way that some folks can create a rich icon set that is at once expressive and still remains cohesive. I especially appreciate icon sets that are freely available, not because I am cheap (though I really am). Rather I like them because they are just easier to use—at the beginning of a project or anywhere thereafter, regardless of whether or not they are open source. It also makes it easier to write about.

I like them so much that I often donate to them without any immediate intention to use them. One such icon set is the Batch icon set. It has been so long since I donated to that one that it took me a minute to remember the name.

Batch, like most icon sets, includes SVG icons. I hope to make use of them as icons in the ICE Code Editor. Specifically, I would like to include an “expand” icon to switch from the embedded to the full-screen version of the editor.

The Batch icon set includes SVGs that are 48px × 48px. I think that is probably too large, so I am going to attempt to scale the SVG. Hopefully this will not be too hard, the “S” in SVG being scalable and all. I add a new method, addControls(), to the ICE.Embedded class and invoke it from the constructor. In that method, I create an SVG element, add a path to it (pulled directly from the Batch icon file), then add the svg to the editor:
Embedded.prototype.addControls = function() {
  var expand = document.createElementNS('http://www.w3.org/2000/svg', 'svg');
  expand.setAttribute('width', 32);
  expand.setAttribute('height', 32);

  var path = document.createElementNS('http://www.w3.org/2000/svg', 'path');
  path.setAttribute('fill', 'rgb(0,0,0)');
  path.setAttribute('d', 'M 39.00,48.00L9.00,48.00 c-4.971,0.00-9.00-4.029-9.00-9.00L0.00,9.00 c0.00-4.971, 4.029-9.00, 9.00-9.00l30.00,0.00 c 4.968,0.00, 9.00,4.029, 9.00,9.00l0.00,30.00 C 48.00,43.971, 43.968,48.00, 39.00,48.00z M 42.00,9.00c0.00-1.656-1.341-3.00-3.00-3.00L9.00,6.00 C 7.344,6.00, 6.00,7.344, 6.00,9.00l0.00,30.00 c0.00,1.659, 1.344,3.00, 3.00,3.00l30.00,0.00 c 1.659,0.00, 3.00-1.341, 3.00-3.00L42.00,9.00 z M 36.00,39.00L27.00,39.00 l 12.00-12.00l0.00,9.00 C 39.00,37.659, 37.659,39.00, 36.00,39.00z M 36.00,9.00c 1.659,0.00, 3.00,1.344, 3.00,3.00l0.00,9.00 L 27.00,9.00L36.00,9.00 z M 12.00,39.00 c-1.656,0.00-3.00-1.341-3.00-3.00L9.00,27.00 l 12.00,12.00L12.00,39.00 z M 9.00,12.00c0.00-1.656, 1.344-3.00, 3.00-3.00l9.00,0.00 L 9.00,21.00L9.00,12.00 z');
  expand.appendChild(path);

  expand.style.position = 'absolute';
  expand.style.bottom = '20px';
  expand.style.right = '40px';
  expand.style.cursor = 'pointer';

  this.editor.editor_el.appendChild(expand);
};
I set the width and height attributes to 32 each in the hopes that this will scale the icon. It does not. Instead, the icon is clipped:



It turns out that I just need one other attribute on the svg element: viewBox. I could have guessed this based on the size of the Batch set, but there is no need. I pull the value directly from the svg file itself:
Embedded.prototype.addControls = function() {
  var expand = document.createElementNS('http://www.w3.org/2000/svg', 'svg');
  expand.setAttribute('viewBox', '0 0 48 48');
  expand.setAttribute('width', 32);
  expand.setAttribute('height', 32);
  // ...
}
With that, I have a lovely icon for expanding the from the embedded to the full screen version:




Day #728

Friday, April 19, 2013

Fresh Perspective on Compiling with ACE

‹prev | My Chain | next›

Writing these posts is often like programming. And not in a good way.

The daily deadline forces me to push through to a solution regardless of propriety. Most days this is a good thing—I have to push myself to learn something that I did not know. Other days, I end up pushing through without taking a step back. Happily there is always tomorrow for that step back. And today is tomorrow.

I ran into a few roadblocks last night as I attempted to bundle all of the JavaScript in the ICE Code Editor. Using the Closure minifier, I hit a few JavaScript warnings. When I googled said warnings, I found similar issues from the ACE code editor project, which I am using as the editor in ICE. Oddly the problems seemed fixed earlier this year, but I bundled ACE less than a month ago.

Not knowing what else to do, I modified my local copy yesterday and moved on. A fresh look today reveals that the fine ACE folks did fix that bit of code way back in January. So, had I stopped to grab the most recent ACE yesterday, I could have save myself some trouble. Fresh perspective would have helped last night.

But fresh perspective today turns up something interesting that I had forgotten about ACE: they have several builds, including no-conflict and no-requirejs builds. They might just solve the other problem that I found last night: difficulty working with a web worker that helped with syntax highlighting.

I start by grabbing the latest vanilla release of the ACE files that I use:
➜  ace git:(api) ✗ ls -1
ace.js
keybinding-emacs.js
mode-css.js
mode-html.js
mode-javascript.js
theme-chrome.js
theme-textmate.js
worker-javascript.js

➜  ace git:(api) ✗ wget https://raw.github.com/ajaxorg/ace-builds/master/src/ace.js -O ace.js
➜  ace git:(api) ✗ wget https://raw.github.com/ajaxorg/ace-builds/master/src/keybinding-emacs.js -O keybinding-emacs.js
➜  ace git:(api) ✗ wget https://raw.github.com/ajaxorg/ace-builds/master/src/mode-css.js -O mode-css.js
➜  ace git:(api) ✗ wget https://raw.github.com/ajaxorg/ace-builds/master/src/mode-html.js -O mode-html.js
➜  ace git:(api) ✗ wget https://raw.github.com/ajaxorg/ace-builds/master/src/mode-javascript.js -O mode-javascript.js
➜  ace git:(api) ✗ wget https://raw.github.com/ajaxorg/ace-builds/master/src/theme-chrome.js -O theme-chrome.js
➜  ace git:(api) ✗ wget https://raw.github.com/ajaxorg/ace-builds/master/src/theme-textmate.js -O theme-textmate.js
➜  ace git:(api) ✗ wget https://raw.github.com/ajaxorg/ace-builds/master/src/worker-javascript.js -O worker-javascript.js
With that, I again run the closure compiler:
➜  code-editor git:(api) ✗ java -jar ~/local/java/compiler.jar \
    --js js/ace/ace.js \
         js/ace/keybinding-emacs.js \
         js/ace/mode-javascript.js \
         js/ace/theme-textmate.js \
         js/ace/theme-chrome.js \
         js/rawdeflate.js \
         js/rawinflate.js \
         js/appcache.js js/ice-*.js \
    > js/ice.js
This is different than last night because I do not have to ignore IE errors. And compilation works. So it seems the fine folks of ACE are really on top of things.

Happily, everything still works. The updated ACE even seems to have fix the minor display issue that I ran into last night. What is not working is the web worker. I still need to place it in the top-level of my web server for ACE to find it when compiled. Even if I try to compile it into the ice.js file:
➜  code-editor git:(api) ✗ java -jar ~/local/java/compiler.jar \
    --js js/ace/ace.js \
         js/ace/keybinding-emacs.js \
         js/ace/mode-javascript.js \
         js/ace/theme-textmate.js \
         js/ace/theme-chrome.js \
         js/ace/worker-javascript.js \
         js/rawdeflate.js \
         js/rawinflate.js \
         js/appcache.js js/ice-*.js \
    > js/ice.js
This still will not work. The bundled worker is ignored and a request is made for worker-javascript.js at the top-level of my web server. I even try switching to the no-conflict/non-requirejs version of ACE to no avail.

In the end, I have to set an undocumented (as far as I can tell) setting for ACE to get this to work:
ace.config.set("workerPath", "/js/ace");
With that, my HTML document only sources a single, large file:
<!DOCTYPE html>
<html>
  <head>
    <title>code editor</title>
    <meta charset="utf-8">
    <link rel="stylesheet" href="editor.css">
  </head>
  <body>
    // Web page content & code here...
  </body>
  <script src="js/ice.js"></script>
  <script>
    new ICE.Embedded(
      document.getElementById('code-001'),
      {preview_el: document.getElementById('preview-001')}
    );
  </script>
</html>
And I get a functioning editor and pane. And it only costs two requests (plus the requests from the code sample in the editor):



Best of all, I do not have to include the web worker in the root level of my web server. I would prefer to be able to eliminate the web-worker request entirely, but I can live with this for now.



Day #727

Thursday, April 18, 2013

Closure Compiling JavaScript

‹prev | My Chain | next›

The ICE Code Editor has gotten quite big. I have no doubt that the code size is large, but the sheer number of scripts that needs to be pulled is has gotten quite out of hand:
<!DOCTYPE html>
<html>
  <head>
    <title>Embedded ICE Code Editor</title>
    <meta charset="utf-8">
    <link rel="stylesheet" href="http://gamingjs.com/ice-beta/editor.css">
    <script src="http://gamingjs.com/ice-beta/js/ace/ace.js" type="text/javascript" charset="utf-8"></script>
    <script src="http://gamingjs.com/ice-beta/js/ace/keybinding-emacs.js"></script>
    <script src="http://gamingjs.com/ice-beta/js/rawinflate.js"></script>
    <script src="http://gamingjs.com/ice-beta/js/rawdeflate.js"></script>
    <script src="http://gamingjs.com/ice-beta/js/appcache.js"></script>
  </head>
  <body>
    // Web page content & code here...
  </body>
  <script src="http://gamingjs.com/ice-beta/js/ice-editor.js"></script>
  <script src="http://gamingjs.com/ice-beta/js/ice-embeded.js"></script>
  <script>
    new ICE.Embedded(
      document.getElementById('code-001'),
      {preview_el: document.getElementById('preview-001')}
    );
  </script>
</html>
So tonight, I hope to bring the Closure JavaScript compiler to bear on the problem. If nothing else, I hope mash everything into one usable file.

I grab, unzip and copy the JAR to a common location:
➜  Downloads  wget http://closure-compiler.googlecode.com/files/compiler-latest.zip
➜  src  unzip ../Downloads/compiler-latest.zip
➜  src  cp compiler.jar ~/local/java 
With that, I am ready to try compiling:
➜  code-editor git:(api) ✗ java -jar ~/local/java/compiler.jar \
    --js js/rawdeflate.js \
         js/rawinflate.js \
         js/appcache.js \
         js/ace/* \
         js/ice-* \
    > js/ice.js
Unfortunately, I get a bunch of errors that prevent compilation. Even more unfortunately, the errors come from the ACE code editor. Some of them are parse errors. I'll deal with them next. The bulk of the errors are Internet Explorer woes:
js/ace/mode-html.js:366: ERROR - Parse error. IE8 (and below) will parse trailing commas in array and object literals incorrectly. If you are targeting newer versions of JS, set the appropriate language_in option.
                token: "string.regexp",
                ^

js/ace/mode-html.js:383: ERROR - Parse error. IE8 (and below) will parse trailing commas in array and object literals incorrectly. If you are targeting newer versions of JS, set the appropriate language_in option.
                token: "string.regexp",
                ^
...
Of course, Internet Explorer is completely incapable of doing any of the stuff in 3D Game Programming for Kids anyway (lacking WebGL among other things). In other words, potential IE problems are not problems at all. So I add the --jscomp_off internetExplorerChecks command line switch:
➜  code-editor git:(api) ✗ java -jar ~/local/java/compiler.jar \
    --jscomp_off internetExplorerChecks \
    --js js/rawdeflate.js \
         js/rawinflate.js \
         js/appcache.js \
         js/ace/* \
         js/ice-* \         
    > js/ice.js
That leaves me with just parse errors. I do not see much choice but to directly edit the ACE code. Happily, this does not involve much. I follow the warning suggestion:
js/ace/ace.js:9122: ERROR - Parse error. missing formal parameter
    this.findMatchingBracket = function(position, char) {
                                                  ^

js/ace/ace.js:9125: ERROR - Parse error. identifier is a reserved word
        var charBeforeCursor = char || this.getLine(position.row).charAt(position.column-1);
And change the char variable name to chr:
function BracketMatch() {

    this.findMatchingBracket = function(position, chr) {
        if (position.column == 0) return null;

        var charBeforeCursor = chr || this.getLine(position.row).charAt(position.column-1);
        // ...
    }
}
That leaves me warnings, but no errors that would prevent compilation. I end up with a pretty hefty JavaScript file:
➜  code-editor git:(api) ✗ ls -lSh js
total 624K
-rw-r--r-- 1 chris chris 469K Apr 18 22:27 ice.js
-rw-r--r-- 1 chris chris  53K Sep  9  2012 rawdeflate.js
-rw-r--r-- 1 chris chris  26K Apr  5 00:43 editor.js
-rw-r--r-- 1 chris chris  20K Sep  9  2012 rawinflate.js
-rw-r--r-- 1 chris chris  18K Apr 15 23:03 ice-full.js
-rw-r--r-- 1 chris chris 5.6K Apr 17 23:44 ice-editor.js
-rw-r--r-- 1 chris chris 5.5K Apr 11 22:53 ice-store.js
drwxr-xr-x 2 chris chris 4.0K Apr 18 22:27 ace
-rw-r--r-- 1 chris chris 3.4K Apr 17 23:46 ice-embeded.js
-rw-r--r-- 1 chris chris  620 Mar 12 22:59 appcache.js
Since ACE itself (in a sub-directory) is 800k+, I think I can live with that. But it will not matter much if the code does not work.

Sadly, the code does not work.

I change the web page so that it only needs to include the compiled ice.js:
<!DOCTYPE html>
<html>
  <head>
    <title>code editor</title>
    <meta charset="utf-8">
    <link rel="stylesheet" href="editor.css">
  </head>
  <body>
    // Web page content & code here...
  </body>
  <script src="js/ice.js"></script>
  <script>
    new ICE.Embedded(
      document.getElementById('code-001'),
      {preview_el: document.getElementById('preview-001')}
    );
  </script>
</html>
But, when I reload, I get an error that the require() function from require.js is not defined:
Uncaught TypeError: Property 'require' of object [object Object] is not a function
My initial instinct is to despair. After a little while, I pull myself together enough to wonder if the order of the ACE files is causing the problem.

So I try being explicit about the order of things:
➜  code-editor git:(api) ✗ java -jar ~/local/java/compiler.jar \
    --jscomp_off internetExplorerChecks \
    --js js/ace/ace.js 
         js/ace/keybinding-emacs.js \
         js/ace/theme-textmate.js \
         js/ace/mode-javascript.js \
         js/ace/theme-chrome.js \
         js/ace/worker-javascript.js \
         js/rawdeflate.js \
         js/rawinflate.js \
         js/appcache.js  \
         js/ice-* \
    > js/ice.js
That changes the error message. But I still get an error in the JavaScript console:
Uncaught atempt to load ace worker into main window instead of webWorker
It seems that, however ACE defines this web worker code, it is somehow dependent on being a separate require.js file. So I recompile again, but without the web worker.

After reloading, I find that the worker library is required, but not at the correct URL:
GET http://localhost:3000/worker-javascript.js 404 (Not Found) 
The URL should include the js/ace sub-path. I will worry about getting that value correct another time. For now, I add the library to the top-level of my web server, reload and get:



Yay! It works.

All is not perfect, however. Moving the web working code to the top-level directory might work on a local server, but it is not an option when I publish this code to http://gamingjs.com. The other problem that I find is that, if I reload the page, the code goes missing:



The code is not actually missing completely. If I resize the browser, it triggers ACE to redraw the code. If I recall correctly, I already have code in ICE to force this redraw on load. Unfortunately, however that works does not account for loading the ICE code along with everything else.

Those two issues aside, this looks to be quite promising. I call it a day here and will pick back up with those two issues tomorrow.


Day #726

Wednesday, April 17, 2013

Really, Really, Really Resize an Iframe Window

‹prev | My Chain | next›

So it turns out that I did not, in fact, solve my problem last night.

I ran into one of those weird browser timing issues that only shows up if you reload the page a dozen times and do a little dance. The problem is with the iframe that holds the animation in my live code ICE Editor. What should look like this:



Ends up looking like:



The problem is that the window inside the iframe somehow ends up with the wrong dimensions. Instead of the smaller 600x300 size that it ought to have, it thinks that is the size of the entire browser's viewport (around 1200x800).

Yesterday I was able to reproduce the problem fairly reliably until I explicitly set the width and height of the iframe that is created to hold the preview:
// Getter for the current preview iframe element. If no preview iframe
// exists, a new one will be created.
Editor.prototype.createPreviewIframe = function() {
  var iframe = document.createElement( 'iframe' );
  iframe.width = this.preview_el.clientWidth;
  iframe.height = this.preview_el.clientHeight;
  iframe.style.border = '0';
  this.preview_el.appendChild( iframe );

  return iframe;
};
After doing this, I boldly claimed that the problem was solved and went to bed. When I woke, saw the problem again. It is harder to reproduce with this code, but I can eventually reload the page enough and do my dance just right. I do see the same behavior.

This is one of those weird cases that do not show up in StackOverflow searches. How many people are dynamically creating iframes so that they can dynamically create the web page inside that iframe? But I need to be able to do this for educational material in support of 3D Game Programming for Kids.

So I fiddle around in the JavaScript console. One of things that I love about the Chrome developer tools is that it autocompletes for all matching properties and methods—even without any letters entered:



Eventually I stumble across the resizeTo() method of a window. It turns out that calling this method on the iframe's window after the iframe has been added to the document clears up the problem:
Editor.prototype.createPreviewIframe = function() {
  var iframe = document.createElement( 'iframe' );
  iframe.width = this.preview_el.clientWidth;
  iframe.height = this.preview_el.clientHeight;
  iframe.style.border = '0';

  this.preview_el.appendChild( iframe );

  iframe.contentWindow.resizeTo(
    this.preview_el.clientWidth,
    this.preview_el.clientHeight
  );

  return iframe;
};
I am almost positive that this resolves the problem. I have reloaded the page many times and done my dance quite nicely. And it always displays correctly. Now all that remains is the sleep test. As long as it works tomorrow, I will consider this good to go.

Update: a sample page with the embedded editor.


Day #725

Tuesday, April 16, 2013

Bugs. Web Bugs

‹prev | My Chain | next›

Some days it really sucks to be a web programmer.

(must… resist… urge… to wax poetic about Dart)

Today is one of those days. I have three different, weird little bugs in the ICE Code Editor. First, the embedded version is not showing code anymore. This seems a simple styling issue so I will work that last. Second, the side-by-side embedded version works, but displays oddly when loaded from cache. Third, the embedded version does not work at all in FireFox. I start with the last and work my way back.

The FireFox issue is that reading from the <script> tag is failing with the error:
Error: TypeError: this.script.innerText is undefined
Source File: http://localhost:3000/js/ice-embeded.js
Line: 45
This is an easy enough fix, because of course I should be using textContent instead of innerText (or innerHTML or text):
// Process the sourcecode from the `<script>` tag. This is ne...
Embedded.prototype.processSource = function() {
  return this.script.textContent.
    replace(/^-(\w+)(.*?)\s*\{([\s\S]+)-\}.*$/gm, "\n<$1$2>$3</$1>").
    replace(/^-(\w+)(.*)$/gm, "<$1$2></$1>").
    replace(/^\s+/, '').
    replace(/\s+$/, '');
};
Naturally, that is not the end of incompatible features: I replace insertAdjacentElement() with instertAdjacentHTML() elsewhere in the code. With that, I have the editor working in FireFox:



It still works in Chrome (since these DOM methods have better support everywhere).

That ended up being easier that I had thought. It was still good to get it done first since this bug would have affected the most potential readers

The next problem is a strange browser cache issue. Under certain circumstances, the preview pane ends up thinking that it is larger than it really is. The result is that the preview no longer fits in the preview area:



I never see this behavior when I first load the page or if I reload the page. I only see it if I return to the page with everything still in cache.

After a bit of investigation, I find that the <div> element that contains the preview layer in the page is not the problem. Nor is the <iframe> that is embedded into that the preview <div>. Both are the correct size (a little more than 600px wide). The problem is only the <canvas> tag that should fit the entire <iframe>. Instead of the 600px width that it should be, it is the size of the viewport:



Interestingly, setting a timeout has no effect. Even if I wait for a full second before displaying the renderer, I still see the same thing. Eventually, I track this down, not to the ICE Code Editor, but to the code that I am setting inside ICE:



For some odd reason, the innerWidth value is wrong for the frame that holds the preview. The code uses this width to set the size of the canvas and mayhem ensues. Actually, this is ultimately an ICE issue. I had been setting the <iframe> width and height to 100%, meaning 100% of the parent tag. But it seems that, when much of the code is coming from cache, Chrome miscalculates the width of the parent tag as being 100% of the viewport.

Anyhow, I have access to the preview element, so I can set the iframe element's width and height to the same value as the preview:
// Getter for the current preview iframe element. If no preview iframe
// exists, a new one will be created.
Editor.prototype.createPreviewIframe = function() {
  var iframe = document.createElement( 'iframe' );
  iframe.width = this.preview_el.clientWidth;
  iframe.height = this.preview_el.clientHeight;
  iframe.style.border = '0';
  this.preview_el.appendChild( iframe );

  return iframe;
};
Problem solved.

Day #724

Monday, April 15, 2013

Excellent Simulations Next to Live Code

‹prev | My Chain | next›

I have the ICE Code Editor embedded into web pages so that the visualization layer can either be behind the editor proper or so that the editor and visualization can be side-by-side. Tonight, I hope to get the side-by-side visualizations working a little better.

And by better, I would like the visualization to remain on the page:



I think that this is likely due to some left-behind hard-coding, so hopefully this will be an easy fix.

This guess turns out to be not entirely accurate. I had left behind some styles on the ICE.Editor class (the combination of the editor and visualization layer). Those styles mostly belonged to the ICE.Full class that is responsible for using ICE.Editor in a full-page IDE-lite.

There was a change needed for ICE.Embedded as well. I add an instance property that describes if the editor and preview layer are overlay-ed on each other. For now, this is only true if a separate, external preview element is supplied:
function Embedded(script, options) {
  this.script = script;
  if (!options) options = {};

  this.overlay = !options.preview_el;
  // ...
  this.applyStyles();
}
Then, in the applyStyles() method, I make several styles conditional on being an embedded editor:
Embedded.prototype.applyStyles = function() {
  if (this.overlay) {
    this.editor.el.style.position = 'relative';
    this.editor.editor_el.style.position = 'absolute';
    this.editor.preview_el.style.position = 'absolute';
    this.editor.preview_el.style.top = '0';
  }
  // Regular styles here...
};
With that, I have all three of the use-cases for ICE working. The full screen IDE:



The embedded editor with code overlaid the preview layer:




And the embedded editor with code and preview separated:



For both the embedded editors, it would be nice to be able to scroll the code to a particular line number. I think it makes sense to be able to specify this on the <script> tag that contains the code code to be embedded and visualized:
<table width="100%">
<tr>
<td width=50%>
<script type="text/ice-code" id="code-001" line="21">
  // Code goes here...
</script>
</td>
<td>
<div id="preview-001"/>
</td>
</tr>
</table>
I do not know the best way to grab arbitrary attributes from DOM elements, but the attributes property seems to do the trick:
Embedded.prototype.attributesFromSource = function() {
  this.line = this.script.attributes.line.value;
};
I can then use this value to tell the ACE code editor to scroll to the supplied line number:
function Embedded(script, options) {
  // ...
  this.attributesFromSource();
  if (this.line) {
    this.editor.editor.scrollToLine(this.line);
  }

  this.applyStyles();
}
That actually works:



That is nice. Now I can control the code that is shown, skipping over the setup code and going directly to the more relevant code for the current discussion.

The this.editor.editor is awkward—the ACE editor of the ICE Editor almost makes me want to switch code editors just to avoid some confusion. But I will add a proxy method to ICE.Embedded so that this.editor.scrollToLine() works. I may even make this indexed from 1 rather than ACE's indexed from zero. All of that is fodder for tomorrow.


Day #723