Processing.js – Irregular Apocalypse

Surprise

Just as aliens seek to strike suddenly and disappear before X-COM arrives, so to does this blog arise inexorably from the deeps to... do something and go... somewhere before... Well, perhaps that wasn't such an apt comparison after all.

Nonetheless, it returns with tardy tidings of the past and elusive glimpses of the future.

The Light of Ancient Days: Release 0.7

0.7 was not particularly exciting. The only real coding I did was for ticket #156, which converted the contents of a file into a byte array (actually an array of numbers). This effectively piggy-backed off of the preexisting ajax helper function, which returned the contents of a file as a string. From there each char was converted into a number via String::charCodeAt().

• #156 loadBytes()

From release 0.6 one may recall how I noticed a little problem concerning the automatic conversion of character arguments into numbers, making it very difficult to pass chars into functions.

Not able to bring myself to let it pass, I took a crack at it. Indeed, the fix seemed simple: remove these two lines from the parser:
// Force characters-as-bytes to work.
aCode = aCode.replace(/('(.){1}')/g, "$1.charCodeAt(0)");
With those lines removed, the tests came back green and it seemed that particular beast was on its way into the cage. But you know what they say about things that seems too easy.

Corban was quick to show that we still needed the auto-conversion facility and if something was to be done, removing those two lines was definitely not the end of the story. Within six hours, he'd come up with a much craftier solution: a Char class. Now, functions could check if an argument was of type Char and act accordingly. Problem solved!

• #312 Char arguments indistinguishable from numbers

Along the way, Anna poked me to take a look at ticket #360, noting that in this example the yellow letters were appearing white. This was due to the char-to-int conversion issue, and when that was solved, #360 was resolved without any additional work.

• #360 basic letters example displays wrong info

All's well that ends well, yes?

A More Recent Note: Release 0.8

The ride on the 0.8 line started innocently enough with the conversion of some visual tests into unit tests. Two failing tests for binary() had to be corrected. One was caused from an incorrect comparison value (Pjs was otherwise returning the right value). The second error was fixed by converting binary() to detect objects of type Char. Along the way boolean() was changed to throw errors on invalid arguments, instead of returning undefined.

• #414 Converting binary() tests from visual to unit causes two to fail.
• #412 Throw errors on invalid arguments in max() and min().
• #413 Convert visual tests to unit tests.

There was a short stop at ticket #269, concerning whether or not a cloned ArrayList should contain deep copies of items contained in the original. Processing uses Java's ArrayList, which does not deep copy. Since Pjs' ArrayList already conformed to this, the ticket was resolved as invalid.

• #269 ArrayList.clone does not deep copy

A rather longer stop was for printMatrix(). printMatrix() outputs the current transformation matrix to the console. Work started in 0.7 but at the time it only worked when Pjs was working in a 3D context. Why? Because Pjs handles the 3D matrix itself, courtesy of PMatrix3D, and so the values of the 3D transformation matrix were readily accessible. When working in the 2D context, Pjs used canvas' internal matrix. This worked very well, except there was no way to retrieve the values of canvas' transformation matrix, and so there was nothing for printMatrix() to print.

• #174 printMatrix()

printMatrix() was ultimately made ready for 0.8, thanks to work on a new and wonderful class...

PMatrix2D

During a phone conference in early March I proposed the creation of PMatrix2D to get printMatrix() going. This meant replicating everything canvas already did just to be able to get the transformation matrix values. I was hoping I was missing something and that there was another, easier way, but apparently there was not. At the end of March the green light was given to cobble together PMatrix2D. Maybe the canvas API will be expanded in the future to allow the values to be retrieved.

Choosing where to start implementing PMatrix2D for release 0.8 was easy: I coded just what was needed to convert the existing PMatrix3D and PMatrix3DStack tests and examples into their 2D equivalents.

Rather than creating a new PMatrix2DStack class, it was easier to transform PMatrix3DStack into PMatrixStack which handled both 2D and 3D matrices.

• #439 Implement PMatrix2DStack
• #428 Implement PMatrix2D::apply(), PMatrix2D::skewX(), and PMatrix2D::skewY()
• #424 Implement Pmatrix2D::set().
• #432 Implement PMatrix2D constructors.
• #437 Implement PMatrix2D::print().
• #431 Implement Pmatrix2D::reset().

Toward Release 0.9 and 1.0

Things seem pretty clear cut from here on in. For 0.9 I hope to finish off implementing the rest of PMatrix2D's functions. 1.0 will aim to get Pjs actually using it.

Processing.js – Some bits for 0.6 in time for midterm season

The Foundations of the Universe are based upon certain Truths, like: as the time to March break/reading week approaches zero, the frequency of exams and due dates sky rocket. Coincidence? There's got to be a formal proof somewhere on the Internet...

But before the coming crunch time gets real bad, is 0.6! From a developer's standpoint, the big improvement for 0.6 is unit testing. Now, just testing return values no longer requires cobbling together a full Processing program, and the tests may be more easily reused and modified by other developers. Yes, this means “pretty pictures" will appear far less frequently now., but on the whole is that really a bad thing? (Using canvas and Processing just to spit out numbers and letters doesn't do justice to either.)

char(), float(), and str()

First up: enhancing a trio of conversion functions. They may now handle the range of primitive type and array arguments they are supposed to.

While the functions themselves don't do anything particularly remarkable, their behaviour helped uncover a bug in Pjs. When user code is parsed, Pjs converts chars into numbers. The relevant code is inside the Processing.parse() function:
// Force characters-as-bytes to work.
aCode = aCode.replace(/('(.){1}')/g, "$1.charCodeAt(0)");

The parser finds single characters flanked by single quotation marks and calls charCodeAt() on those characters. What this means is overloading Pjs functions to handle both arguments of type Number and characters (essentially one-character Strings in JavaScript) becomes... difficult.

Take str('A'). This should return the string “A". But because of the parser what actually gets processed is str(65) (the number 65), returning the string “65". Without jumping additional hoops, you can't tell str() whether that 65 is supposed to be a number 65, or if you actually mean that 65 to be representative of the char 'A'.

This causes some str() tests to fail. char() tests are affected although they all pass. For example:
int i = 65;
_checkEqual('A', char(i));
This is evaluated as _checkEqual(65,65), not _checkEqual('A', 'A'). The parser's fudging is masked by the passing tests.

I've filed a ticket with the suggestion that the parser not convert chars to Numbers, and leave functions to do this by themselves. With unit testing now rolling it'll be easier to find which functions rely on the parser's current behaviour. I may very well pick up the ticket for 0.7.

• #303 char() suppose to work on all primitive datatypes including arrays
• #302 float() suppose to work on all primitive datatypes including arrays
• #304 str() suppose to work on all primitive datatypes including arrays

normal()

normal() takes three numeric arguments representing a vector and saves them to global variables. Oh, and it changes a flag. All of these (variables and flag) are supposedly used by other functions, but none of those uses have been implemented yet. (“What came first, normal() or the code that uses normal()?")

Since normal() has no return value, the unit tests content themselves with checking the correct number and type of arguments are present.

• #125 normal()

smooth()/noSmooth()

char(), float(), str(), and normal() are the good news. Alas, smooth() and noSmooth() fall somewhat short of that mark and probably won't appear in 0.6.

smooth() and noSmooth() are intended to toggle anti-aliasing on and off when painting. Canvas has no built in facility for this. sephr found a Mozilla-only CSS property image-rendering applicable to the canvas tag, and perhaps able to achieve what was desired.

Unfortunately, while image-rendering is good for raster images it doesn't work the same magic on canvas drawings. For the time being, it seems smooth() and noSmooth() will go into hibernation.

• #145 smooth() and noSmooth()

Processing.js - It's a long way to 0.6

“But my heart's right there", or so the song goes. Indeed, it looks like my work is laid out for me. So for this short update, I'll just list what I hope to get done for February 19th (in no particular order):

• Port normal() (Ticket)
• Complete porting smooth() and noSmooth() (Ticket)
• Upgrade char(), float(), str() to accept a multitude of argument types (Ticket)

This will also be the first release where we'll be using automated tests! I wonder if I still have Minefield stashed away somewhere...

Processing.js – int() and Release 0.5

int(), the late passenger

A long time ago, in the forest across the river, where fiery leaves danced upon the crisp winds of pseudo-winter (or in other words: back in the 0.2 release) I finished porting boolean(), a function that converted primitive types to either true or false, over to Processing.js (Pjs). As it transpires, boolean() was just one a gaggle of conversion functions, many which continue to seek the company of a code monkey willing to give them a boost over the wall from Java-land into JavaScript-land.

Yesterday evening, at the behest of The Powers That Be, and against the questionable judgement of my lazy self, I tentatively picked up the rest of these converters. And as I told Anna, the recently promoted Fearless Leader, I'd try to get one of them done for 0.5, whose deadline is today. So, without further ado, I present to you int() for Pjs.

Who wants to write the thousand words?

Lo, see now int() in action in Pjs, and compare it to how the same code renders in Processing:



Your eyes do not deceive you: Pjs' int() fails a test. But don't break out the torches and pitchforks just yet! There's a good reason for this. int() may take a byte or an array of bytes as an argument. In Processing, a byte may have a value from -128 to 127, but more importantly: byte is a static type. Through overloading this allows Processing methods, written in Java, to behave differently to accommodate different variable types.

Things are not so simple in JavaScript. In Pjs a Processing byte and int gets converted into a JavaScript 'number'; effectively there is no distinction between a byte and an int in Pjs. Since there's no distinction, int() cannot be tailored to behave differently when it receives bytes. (floats and doubles are different since you can always check if there's a decimal point in there somewhere.) Therefore, the test fails.

A possible solution may be to add a second parameter to int(), a flag called 'isByte' or something. That might be something to bring up in next week's telephone conference.

int() links

• Lighthouse ticket #220 int(), float(), str(), et al are suppose to work on all primitive datatypes including arrays
• GitHub commit

A Quick Recap of 0.5
Just for the record, here are the links to the rest of the stuff I did for 0.5:

• Enhance existing min() and max() implementation
• Implement String::equals()
• Enhance existing nf() implementation

I'll be making a (hopefully) shorter post soonish outlining what I hope to accomplish for 0.6. But for now, it's time to switch gears and call it a morning.

Processing.js – nf()

Closer to release 0.5

With a little luck, this should be my the last piece for the 0.5 release of Processing.js (Pjs) . The nf() function should now be able to handle arrays of numbers to be formatted, and float formatting.

There's a few interesting things going on here that the Processing documentation doesn't allude to, but those are more easily revealed once we see the code in action.

A few pixels

Here's the first batch of tests in Pjs (hah, there's no way I'm trying to minify that for the post!) and here's what Processing generates for the same code:



And here's the second batch of tests (arrays) in Pjs, and here's what Processing generates for the same code:



Note the tests use String::equals() which I recently completed and is not yet in the public release.

Also, while the documentation says all arguments should be positive integers, Processing's nf() does handle arguments with values zero or less. So I've coded Pjs' version to do the same.

It's supposed to do that? behaviour.

There are two weirder-than-normal things going on.

In the tests 19 to 32 in the first batch you'll find that depending on whether the one's digit is odd or even, the 'rounding' is different. This required a bit of extra code to replicate.

In the second batch, nf() processes values in the int array “incorrectly"; running through the ints individually results in the expected results. Strangely enough, my code was able to replicate this behaviour right off the bat.http://wobblyretroaction.blogspot.com/2010/01/processingjs-stringequals-or-how-i.html

Links

• Lighthouse ticket #227 nf()
  
• GitHub commit

Processing.js – String::equals() (or: how I filled something in on the way)

Pothole? What pothole?

Open source development is like driving on a road with potholes where every driver also happens to be a road worker. One day you may just get fed up with driving over the same crater over and over again and decide to do something about it in the most direct way possible: stop the car, get out, and fill it in yourself.

Of course, some holes get away for a while because they're just not that bad, or too deep. Other's scream for immediate rectification. Like the maws that open up suddenly and force you to slam on the breaks less you go careening into the deeps of the Earth. Or the holes just big enough to swallow a wheel and bring the scheduled excursion to a halt.

Well, on the way to giving nf() a nudge I discovered one of those wheel eaters. Like Java, Processing uses an equals() method to compare two Strings. JavaScript gets away with just ==, and for the past while Processing.js (Pjs) has done without equals() and used just ==. This was all well and good... until I started writing tests for nf().

The nf() tests compare the String result of nf() and an expected String result. Depending or not the two Strings are the same the output's colour changes. To ensure the code used for the test in the Processing Development Environment and in Pjs were the same I needed equals(). (Mind you, I could have used == in Pjs, equals() in Processing, and get the same output, but that would have been cheating.)

So with that short story aside, I present a quick and dirty String::equals() for Pjs. May this hole never become deeper.

How the road looks now

So here's how it looks in Pjs. (Again, we'll have to make due with a link. Minifying the code for insertion into the post didn't work out to well.) This example is extended from the Processing example.

And here's how the same code looks like in Processing:



Links

• Lighthouse ticket #246 "str1".equals("str2") function
  
• GitHub commit

Processing.js – Improving min() and max()

And another brick

So here's another little improvement for Processing.js.

In Processing, the min() and max() functions may take 2 or 3 numeric arguments, or one argument if its an array of numbers. For a while the Pjs implementation could only take two arguments. I have since taken it in hand and the Pjs implementation should now be able to do what the Processing implementation can do.

Well, not exactly. In fact, the Pjs implementation will happily take a great many more numeric arguments than just three. It'll even take one number in a pinch. Passing in non-numeric arguments, aside from a single array, will cause "undefined" to be returned. Trying to be clever and pass in an array as one of two or more arguments will also return "undefined". In Processing these "undefines" would be "thrown exceptions".

My eyes!

This will be another post where I'll provide links to the Pjs tests and comparison Processing images.

Basic Processing example for min() in Pjs, and how it looks in Processing:



Basic Processing example for max() Pjs, and how it looks in Processing:



3 and 4 numeric arguments, and argument type discrimination, for min() in Pjs, and how it looks in Processing. Note that only three arguments is tested in Processing since the rest cause exceptions.



3 and 4 numeric arguments, and argument type discrimination, for max() in Pjs, and how it looks in Processing. Note that only three arguments is tested in Processing since the rest cause exceptions.



For the tests I had to pull a patch or two fixing Ticket #57 Boolean array bug.

Links

• Lighthouse ticket #185 The min() and max() functions effectively take only two arguments
• GitHub commit

DPS911 Project – Processing.js – v0.4

A bit of touching up
So here we are: the first release for DPS911 this semester! For this release I've added missing functionality to Processing.js's (Pjs) implementation of text(). Unlike before, the function may now handle the printing of variable of integers, floats, doubles, and bytes, in addition to strings.

Now I wait for a (hopefully) favourable peer review.

Pretty pictures
Here we see a live demo of how it works in Pjs now as shown on this page:



And here we see the same demo as displayed in the Processing Development Environment:


What are we looking at?
The Pjs implementation had to take a few things into account. Processing displays floats and doubles to the thousandths digit, which is easy enough to replicate in JavaScript using toFixed(). But notice how Processing rounds: for 2.7185 it rounds down to 2.718 instead of up to 2.719.

toFixed(), by itself, rounds 2.7185 to 2.719, so a little bit of extra code was needed to alter this, like so:

if ( (val*1000) - Math.floor( val * 1000 ) == 0.5 ) {
  val = val - 0.0001;
}
val = val.toFixed(3);

So for 2.7185 we craftily modify it to 2.7184 before sending it to toFixed().

Pertinent links

• Lighthouse ticket (with patch)
• GitHub repo commit where it comes together

Thoughts on the Proper Care and Feeding of Git Repos on GitHub

The nature of the problem

Last semester I and the other students working on Processing.js (Pjs) started using GitHub. We all made our own branch of the Pjs trunk repo and started hacking away. Over time we committed to our branches and some of our improvements went into trunk.

But our very success started to have some unintended consequences. Trunk advanced but our branches did not gain the improvements merged into trunk. This made our antiquated branches increasingly poor bases to produce patches for trunk. Needless to say, this inconvenience was not appreciated by anybody.

After a bit of experimentation I managed to find a way to keep my branch up to date without deleting it and starting fresh. Mickael Medel (aSydik), one of my classmates, has requested I spread the joy and blog about what I did. So here it is.

Fractional-reserve coding

In five easy steps most, if not all, problems, may be averted.

Step 1 – Pull from trunk to the branch on your local system

Before making any changes locally make sure you have the latest version of trunk. Pulling from trunk every time it changes isn't such a bad idea: it's much easier for Git to automatically merge code when the difference is small or simple. If the difference is too complex Git may cry "conflict!" and make you sort out the mess. Not good.

For us monkeys working on Pjs, I believe the pertinent pull command is:

git pull git://github.com/jeresig/processing-js.git master

Now, if you have more than one development system (as I do: desktop at home, and laptop) you may need to pull from your own repo instead of trunk. In that case the pertinent command is something like:

git pull git@github.com:yourusername/processing-js.git master

Step 2 – Hack on your local branch

So now have a totally updated version on your local system and the time has come for you to work your magic. Hack, surf, and make merry!

Step 3 – Commit on your local branch

After many hours of fruitful hacking you look out the window and the sun has been suspiciously replaced by the moon. It's time to call it a morning and push things back to your GitHub repo. But wait! Remember to commit the changes first, otherwise you'll have nothing to push.

I guess that Step 2 and 3 are just reminders, right?

Step 4 – Push your local commits to your online repo

Now comes the culmination of your efforts, fuelled by countless grams of caffeine and other tasty goodies: the big push. At this point you need to update your online repo to match the local repo. Not to hard, just plug in something that looks like this:

git push git+ssh://git@github.com/yourusername/processing-js.git

If all goes well the local commits will flood across cyberspace, and you shall know balance and harmony...

Step 5 – Back to Step 1

… for a little while. When you're ready to go at it again, collect your $200 and sally back to Step 1.

Opposites attract, but sometimes we wish they wouldn't

The above works very well if your branch is kept up to date. But if you're sitting on some really old revision chances are Git will be unable to do the merge in Step 1. It'll cry "conflict!" and you'll have to sort it out. By this I mean you'll need to do the merge manually (yes, lots of copying and pasting and what not.)

If the situation is really bad you may simply want to kill and remake your branch from scratch, keeping in mind to regularly update the resuscitated branch this time round.

If you slog through the manual merge, then commit changes to finish the merge. Committing will also tell Git the conflict has been resolved and you'll be back in business. At least, that's what I think happened when I resolved my first (and so far only) conflict.

Only now, at the end, do you understand.

I hope that was helpful for those in need.

I apologize to Mickael for the tardy post. He asked for this last Thursday and it took me a while to cobble this together. You see, I use an unholy combination of the command line and the EGit plugin for Eclipse to interface with GitHub so I wasn't quite sure what the exact commands were. I tested them with my latest commit to GitHub so I hope they work for everybody else.

Yes, I use Eclipse to hack Processing.js, but only because EGit appeals to my laziness. Otherwise I'd do what sane people do and use a "regular" text editor.

2010, DPS911, and more Processing.js!

The Times They Are a-Changin'
Last semester in 2009 (so long!) I wandered into DPS909 and busied myself porting some functions over from Processing to Processing.js (Pjs).
For more on what Pjs is about check out a previous ancient post.
In a fitting continuity of events , this semester in 2010 (hello!) I've wandered into DPS909's follow-on, DPS911 and will continue futzing around with Pjs. The work done for DPS909 were released as 0.1 to 0.3, so not surprisingly the work for DPS911 will be released as 0.4 to the big 1.0.

The Plan
For v0.4 to v1.0, I hope to port over some new functions in addition to fixing existing bugs. The prospective task list and timeline is given below:

• Release 0.4 – Jan. 22, 2010
  • Fix text() to accept non-String arguments.
• Release 0.5 – Feb. 5, 2010
  • Fix min() and max() to take more than two arguments each.
• Release 0.6 – Feb. 19, 2010
  • Port loadBytes().
• Release 0.7 – Mar. 13, 2010
  • Port normal().
• Release 0.8 – Mar. 27, 2010
  • Port printMatrix().
• Release 0.9 – Apr. 9, 2010
  • Port printCamera().
• Release 1.0 – Apr. 23, 2010
  • Padding for overruns and to give time to fix any bugs from releases 0.1 to 0.3 from DPS909.
  • If that happens not to be necessary, I'll just choose another function to port of bug to fix.

Obligatory List of Relevant Links

• Processing.js @ Lighthouse
• Processing.js @ zenit.senecac.on.ca
• My Processing.js Project Page @ zenit.senecac.on.ca
• My Processing.js repository @ GitHub

DPS909 Project – Processing.js – v0.3

And It's A Warp...
… for this phase. This is the last release for this semester's DPS909 project. For this bit strokeCap() and strokeJoin() were ported from Processing to Processing.js. The meat of what was done may be found in previous posts linked to above, so I won't occupy everybody again by repeating it here (after all, it's exam time!)
So without further ado, here are pertinent the pertinent links:

Links to Elsewhere

• Lighthouse ticket for strokeCap()


• Lighthouse ticket for strokeJoin()


• Code branch at GtiHub

Links to the Past

• Release 0.1


• Release 0.2

And Away!
That's it for now. With a little luck I'll see you right back here in a months time for DPS911.

Processing.js – strokeJoin()

Copy and Pasting,,, Works!
Hot on the heels of strokeCap() strokeCap() is strokeJoin(). This blazingly fast turn around was accomplished by the time honoured method of shamelessly lifting previous work wholesale and tweaking.
Observe: here we have strokeCap():

p.strokeCap = function strokeCap( set ){
if ( set == p.ROUND ) {
curContext.lineCap = 'round';
} else if ( set == p.SQUARE ) {
curContext.lineCap = 'butt';
} else if ( set == p.PROJECT ) {
curContext.lineCap = 'square';
}
};

And here we have strokeJoin():

p.strokeJoin = function strokeJoin( set ){
if ( set == p.MITER ) {
curContext.lineJoin = 'miter';
} else if ( set == p.BEVEL ) {
curContext.lineJoin = 'bevel';
} else if ( set == p.ROUND ) {
curContext.lineJoin = 'round';
}
};

As we can see strokeCap() gained a very close sibling in strokeJoin()!

Newborn Boo-boos
This is not to say the birth of the younger didn't affect the elder. Originally the constants for strokeCap() and strokeJoin() were defined separately, like so:

// Stroke cap constants.
p.ROUND = 0;
p.SQUARE = 1;
p.PROJECT = 2;

// Stroke join constants.
p.MITER = 0;
p.BEVEL = 1;
p.ROUND = 2;

p.ROUND was redefined for strokeJoin() and ended up conflicting with strokeCap()'s p.PROJECT. This caused... incorrect behaviour in an otherwise well behaved strokeCap().
The problem was solved simply be consolidating the constants into:

// Stroke cap and join constants.
p.ROUND = 0; // Used by both cap and join.
p.SQUARE = 1; // Used by cap.
p.PROJECT = 2; // Used by cap.
p.MITER = 3; // Used by join.
p.BEVEL = 4; // Used by join.

Pretty Pixels!
To end off here is the code in action compared to the results of the same code in the Processing Development Environment (PDE).



On a side note I've noticed the Processing website's examples are imprecise; their code and images do not match up. The images show the result of strokeCap(SQUARE) or strokeCap(PROJECT) being called, but these calls are missing in the example code.

Processing.js – strokeCap()

strokeCap()
Another week or two, and a bit of work toward the 0.3 release of my DPS909 project. This time it was strokeCap()'s turn to be ported to Processing.js (Pjs).
So here is the basic Processing example working in Pjs:


And here is the example running in the Processing Development Environment (PDE):

Squishy Innards
This is all done by setting canvas' lineCap attribute. lineCap has three possible values which correspond directly to those used by Processing so strokeCap()'s code looks just like this:

p.strokeCap = function strokeCap( set ){
    if ( set == p.ROUND ) {
        curContext.lineCap = 'round';
    } else if ( set == p.SQUARE ) {
        curContext.lineCap = 'butt';
    } else if ( set == p.PROJECT ) {
        curContext.lineCap = 'square';
    }
};

lineCap is also handled by canvas' state saving functions, which means Pjs' pushStyle() and popStyle() can already handle saving and restoring the setting as well.

A Different Starting Point
The main difference between canvas and Processing/Pjs is the default setting for lineCap. Canvas' default is “butt", which is “SQUARE" in Processing. Processing needs to default to “ROUND", which is “round" in canvas.

There was a bit of trouble getting the default working in Pjs. At first adding curContext.lineCap = 'round'; to function init didn't work. Al MacDonald (F1LT3R) was very quickly able to find where the problem was and fix it: it turns out function size recreated the canvas context every time it was called, which in turn reset all the attributes. Thanks, Al!

So now the default setting works very nicely in Pjs:


Compare to the same thing in PDE:

Useful Links

• GitHub commit where strokeCap() works


• Lighthouse ticket (will need to be reopened for code review)

Firefox Addon and XPCOM

So last week in DPS909 we were supposed to feel our way through a lab giving some idea how Mozilla's XPCOM works. XPCOM is used to ensure code portability. I do not pretend to understand how it all works, it's rather complicated and terrifying for a newbie, but at the very least I can follow instructions.

The big first step is to create an XPCOM interface using a XPIDL file, which includes function prototypes using familiar C/C++ data types (at least in the walkthrough, XPCOM may be used with other languages to). A header file is automatically generated which, amongst other things, transforms the data types into Mozilla's data types and creates a stub class using the header. In other words, a lot of the basic set up work is done for the code cruncher (winner!)

Most of the walkthrough was pretty straight forward. There were a few minor things that needed to be changed or were different. In install.rdf, I changed em:minVersion and em:maxVersion to 3.5.5 and 3.7a1pre respectively (that allowed the extension to work in the trunk build.) Also, IFirstXpcom,h was created in $(objdir)/dist/include/ rather than $(objdir)/dist/include/firstxpcom/.

But everything worked out and so I can now claim to have created (after near slavishly following somebody else's instructions) an add-on for Firefox.

Processing.js – Something Lost/Gained in the Translation

The nice thing about porting is the end result is already known. Comparing the effects of the original with the effects of the transcription is a good indicator of how close you are and whether or not you're on the right track.

Simple, yes?

Sure, at least in a perfect world where I could have my kitchen faucet yield hot chocolate and have my computer magically upgrade itself to the latest hardware and software. Alas, a perfect, honest-to-the-source, zero deviation transcription may not always be possible; such is what I have discovered while tinkering with pushStyle() popStyle() and boolean().

Below you will find I have not embedded the Pjs examples, instead I shall provide a link to another web page with the example on it. This is because embedding requires minification, and I'd rather not go through the frustration of minifying some of the larger examples. I'll provide screenshots of how the examples look using the Processing Development Environment (PDE) for comparison.

Differences in pushStyle() and popStyle()
The basic Processing reference example looks like this

and can now be reproduced in Pjs (Pjs demo here.)

But take a Pjs demo like this one. See the extra pops at the bottom? You can't do that in Processing. The PDE won't allow the Pjs example code to run; it'll see the extra pops, throw an exception, and refuse to run properly until the extra pops are removed. The closest approximation to the Pjs example Processing will allow looks something like this:

Pjs doesn't really concern itself with catching logical errors in the code. At best it just concerns itself with syntactic errors, and if there are any it simply doesn't paint anything onto the canvas. Saying that, the Pjs version of popStyle() has to be a bit more lax than its Processing counterpart and work with, rather than crash on, extra pop calls.

In this matter the solution is simple. Upon figuring out the pop call is extraneous, Pjs effectively ignores it and carries on. It must be one of the few times burying one's head in the sand actually makes the problem go away!
Differences in boolean()
Moving onto boolean(). Let's start out with showing a basic example based on the one found on the Processing website. It looks like this:


and it too may be reproduced in Pjs (Pjs demo here.) However, boolean() is afar more complicated beast than the basic example would imply. In this case it's a matter of how Java/Processing is very strict on the number and nature of arguments passed to a method, while JavaScript/Pjs is not.

Take for example the following this Pjs demo. Note how we're passing in doubles, booleans, ints, nulls, undefines, etc., and arrays with mysteriously empty elements. If you were to try the code in Processing it wouldn't work. For one, Processing's boolean() refuses to handle boolean variables, or double variables, or nulls, and other fancy things like that. In fact, the closest Processing example to the Pjs demo is rather more sparse and looks like this:


(Un)fortunately, Pjs' boolean() can't take the easy way out and throw away arguments it doesn't like, coughing up an exception, and making the whole show grind to a halt. Being JavaScript it's syntactically correct to pass a cornucopia of dynamically typed arguments into a function. Pjs' boolean() has to handle whatever comes its way gracefully.

As you can tell from the Pjs boolean() demo, I thought this was best done by having the function return "meaningful" results from arguments not normally handled by Processing's boolean(). Numeric arguments are handled like integers: zero values return false, non-zero values return true. Boolean arguments are just returned as themselves. Nulls and undefines return false.

Heresy!
I like to think I've cobbled together an acceptable compromise between how Pjs and Processing behave. I guess I'll find out soon whether this is acceptable or not when I try to push the code into the library.

DPS909 Project – Processing.js – v0.2

Returning to Ground Zero
First, a self-plagiarized recap. This semester at Seneca College I am taking a course revolving around open source development. As part of the course the students (yours truly being no exception) were nudged into getting involved in some Mozilla related projects, produce code for the real world, and see fist hand how the whole open source thing works.

I chose to work on Processing.js (Pjs), a JavaScript port of the Java-based Processing. More specifically, I volunteered to work on porting over the functions pushStyle(), popStyle(), and boolean().

On 19 October 2009, I threw release 0.1 out into the cruel world to fend for itself. Today I present release 0.2 to the world. Applause!

Not new, but shinier!
The watchwords for release 0.2 were: fix it. Release 0.1 included reasonably functional code for pushStyle() and popStyle(), and totally untested and untried code for boolean().

Bringing boolean() up-to-spec was the goal of this release and this was done by throwing together some test cases for boolean(). Bugs were indeed caught, squashed, and flushed down the toilet.

Although boolean() got most of the attention, the elder siblings did not totally escape notice. pushStyle() and popStyle() received their own test cases too. One issue was whether the pop should spit out alert popups if there were not enough pushes. I decided not to use any popups and just have a warning be sent to the console, where most users probably won't notice. pushStyle() and popStyle() were further improved to better handle the library's own variables; before they only managed canvas' variables with any proficiency.

Better to get it when it's pleasantly warm rather than skin-charring hot

• Download the full Pjs library (my patched version) OR


• just download the patch

What now?
Pursuant to getting my code merged into the Pjs library there are a few instructions I have to follow. So I'll make another post comparing the results of my Pjs code with those of the Java-based Processing code. As I have discovered it is not totally possible to faithfully port Processing to JavaScript. For example, Processing will throw exceptions and refuse to do anything if you try to pass an argument to a function and the function is not designed to accept, say, the variable type of the argument. But in Pjs you have to handle these weird arguments and preferably return some meaningful result. This comes up a few times in boolean(). But more on that soon.

For release 0.3, it looks like I'll have to choose another function to port over, or at least get started on even if I don't finish by the end of the semester. strokeCap() looks appropriately involving.

Learning a Bit of JavaScript

This week in DPS909 we talked a bit about JavaScript. This was particularly relevant to my project this semester, implementing a few functions in the Processing.js JavaScript library. As part of this week's journey of the mind, we were directed to watch a lecture by Douglas Crockford called "JavaScript: The Good Parts". Leaping over to Wikipedia for some familiarization, Crockford is the senior JavaScript Architect at Yahoo!, and wrote the specification for JavaScript Object Notation. The lecture in the video is based off his book of the same name.

Right off the bat, I'm one of those people who, as Crockford' puts it, started tinkering with JavaScript without learning it. So given my novice credentials coupled with a pressing need to "learn more" I'm pleased that the lecture had a few "do and don't" pointers that someone like me could take away.

The first big thing I learned was use ===, avoid ==. I was first exposed to === by David Humphrey, who gave me a code snippet demonstrating how to detect if a JavaScript variable was an array. A bit of research revealed === compares not only the value of two variables but their types as well, a very useful feature in a language with dynamic typing. I was, not, however, inclined to use === near exclusively.

After watching Crockford's lecture made me realize using mainly == would eventually cause grief.. Sooner or later I would make compare two values, forget the variable types, or forget the distinction between null and undefined, and end up with an annoying, very small, difficult to track down, bug. Using === would be an easy way to save a bit of heartache in the future, which is always a good thing.

Learning the distinction between null and undefined was an ancillary to the first major thing I picked up. The former is a value for an object variable, the latter can be had if trying to retrieve a variable an object doesn't have. Note to self: further research required.

The second, and most surprising, thing I learned was coding style matters. It's easy to mistake JavaScript for a free-wheeling language: dynamic typing, you don't have to use semi-colons half the time! But apparently something as "simple" as bracket placement when defining blocks can really mess things up. Crockford gave the following example:

return 
{
 ok: false
};

This seems innocent enough, but apparently behind-the-scenes shenanigans means it actually is interpreted like so:

return;
{
 ok: false;
}

In other words, everything after return goes nowhere and does nothing. For this to work as intended the opening bracket must be placed to the right of return like so:

return {
 ok: false
};

While my coding style generally resembles the "correct" example, this will be a useful tidbit when deciphering the code of other programmers.

Fortunately, it seems Crockford has developed JavaScript code quality checker called JSLint to help developers catch some of their more egregious code. Methinks I shall be availing myself of it in the near future.

Standing On Their Shoulders: A Short Processing.js Tale

So this semester's reading week has come and gone and once again, as is the familiar post-mortem assessment, I have discovered the ratio of “useful" to “superfluous" work done is heavily biased toward the latter.

On the Processing.js (Pjs) front we have more baby steps!

Don't mind if I do...

For pushStyle() and popStyle() I've come full circle. First I fiddled around with storing and retrieving variables in the library, then I moved onto storing and retrieving canvas variables, and have since returned to working on the Pjs library variables.

In my previous post, I discovered some code by Andrew Peace. As it turns out, not only did Peace's work handle the Pjs variables, he also did it in a more JavaScript savvy way then I would have done.

The example that sticks out in my mind was his handling of arrays. While I was busy incrementing and decrementing a counter to figure out how many states I had and which element to store and retrieve from, Peace simply used the push() and pop() functions that grace all JavaScript arrays.

I have incorporated Peace's code with only cosmetic changes, and JavaScript's push() and pop() functions for arrays has since entered my corpus of All Knowledge. My hat off to Mr. Peace!

But does it work?

In my previous post I also intimated that some work was going on with boolean(). There was some code but no testing. In the last little bit I've started cobbling together a few test cases, and even thought still incomplete they've already helped flush out a few bugs.

The nice test page layout was provided by classmate Daniel Hodgin. Thanks!

DPS909 Project - Processing.js – v0.1

Last month on [insert name of media production here]...

First, a brief recap. This semester at Seneca College I am taking a course revolving around open source development. As part of the course the students (yours truly being no exception) were nudged into getting involved in some Mozilla related projects, produce code for the real world, and see fist hand how the whole open source thing works.

I chose to work on Processing.js (Pjs), a JavaScript port of the Java-based Processing. More specifically, I volunteered to work on porting over the functions pushStyle(), popStyle(), and boolean().

I've got to release something...!

Today I get to make my first (course mandated) release! And this is what it does (trotting out this little “demo" for the third time):




What's that? That, fine feathered friends, is pushStyle() and popStyle() working just a little bit. Nothing particularly amazing, it's just the Processing example copied verbatim.

A start on the boolean() code is also in the release, but it's not tested at all.

For those of you who want to see for yourself may:

• Download the full Pjs library (my patched version)
OR

• Download just the patch

Toward 0.2

The march to enlightenment continues, and there is still much to do. The implementation of pushStyle() and popStyle() remains incomplete as demonstrated by this discussion on the Pjs Google Group. So far I have only saved the state of canvas variables, and not those in the library itself.

Then I need to see about putting through a more expansive test regimen. And there's already a bug: using Andrew Peace's test case, the “more pops than pushes" error alert just keep popping up. I like to think the intended behaviour is for a finite number of warnings.

On the other hand, boolean() seems much more straight forward to test.

A not-so-secret society

v0.1 releases just in time for this years Free Software and Open Source Symposium hosted by Seneca College. This is particularly important because Al MacDonald, Pjs project leader, is attending FSOSS and will no doubt want to have a few words with those DPS909 students working on Pjs.

David Humphrey suggested us students might want to look into setting up a Pjs bug tracker, discuss ways to compile our seperate test suites into one big suite, and perhaps set up a website to showcase the newly ported capabilities. I figure even if I somehow manage to get my project done way ahead of schedule there will not be a lack of things to do.

Fixing a Bug In A Harmless Test Environment

This week's DPS909 lab brought together everything covered thus far in the course: build the software, file a bug, find the offending code, make a patch. If this were any other course this lab might qualify as a "quest": not a quiz, not a test, but somewhere delightfully in between!

Step one was compiling a debug build of Thunderbird/Shredder. I checked out straight from the repo rather than use the code provided by the lab instructions. The process was similar to building Firefox/Minefield and went off literally without a hitch. A very palpable hit for me!

Nest was to confirm the bug's existence. The bug we were interested in deals with Thunderbird converting a string that looks like an email address into a hyperlink; the problem is the string does not represent a valid email address and so should not be converted. We can see the fubared behaviour below:


So having confirmed the existence of the bug without difficulty, bug #7243 was filed in the landfill. I like to think I included the bare-bones information in the description answering the questions of: "what is wrong?", "how may it be reproduced?", and "what version of the software are you using?".

Now came the matter of tracking down that needing futzing with. This took a while. I spent much time plopping in search terms into the Mozilla Cross-Reference without success. I looked into "parse message", "save draft", and the like to try and get into where the message was stored and perhaps displayed to the screen.

In desperation I went back to the program and, mousing over the offending email link, saw it was a mailto link. So I through in mailto as the search term and... there was light. (Just goes to show simplicity is golden.) Conveniently the very first result from that search took me exactly where I needed to go.

if (inString.FindChar('.', pos) != kNotFound)

The conditional only checked if a "." was found after the @ sign. What it also needed to do was check if there were any ".." after the @ too, which would mean it was not a valid address. Some way needed to be found to find whether a ".." was in the string. Unlike what was currently in mozTXTToHTMLConv.cpp I needed a way to check the presence of a string in the string, and not just a char in the string. According to the Mozilla internal string guide the correct way to find if a substring existed was to useFindInReadable in nsScannerString.cpp, However converting the code to use the newer approved way was probably a bit beyond what I wanted to delve into for this lab.

So I just stuck with the old way. Searching through the documentation some more I found nsDependentString had just the function I needed. The code mozTXTToHTMLConv changed to:

if (inString.FindChar('.', pos) != kNotFound && inString.Find("..", pos) == kNotFound)

A recompile later the situation was much improved:


I created the patch from ./comm-central/mozilla and ran it through JST Review. A few "longer than 80 characters" and extra whitespace/tabs warnings later, it was ready to be attached to the bug report.

Ehren graciously took some time to review the patch and found I had indented a line a bit too far. It was up to a patched patch to pass muster instead.

The tool-tips for the review options for attachments threw me off a bit. Super-reviews are only needed for world-shaking changes, but the super-review tool-tip says something somewhat different. I didn't request a super-review for the revised patch.

The patch and review comments have been copied to my Seneca College wiki page.