Unobtainium Base Material

Processing.js – Irregular Apocalypse

2010-04-02T20:45:00.000-07:00

Surprise

Just as aliens seek to strike suddenly and disappear before X-COM arrives, so to does this blog arise inexorably from the deeps to... do something and go... somewhere before... Well, perhaps that wasn't such an apt comparison after all.

Nonetheless, it returns with tardy tidings of the past and elusive glimpses of the future.

The Light of Ancient Days: Release 0.7

0.7 was not particularly exciting. The only real coding I did was for ticket #156, which converted the contents of a file into a byte array (actually an array of numbers). This effectively piggy-backed off of the preexisting ajax helper function, which returned the contents of a file as a string. From there each char was converted into a number via String::charCodeAt().

#156 loadBytes()

From release 0.6 one may recall how I noticed a little problem concerning the automatic conversion of character arguments into numbers, making it very difficult to pass chars into functions.

Not able to bring myself to let it pass, I took a crack at it. Indeed, the fix seemed simple: remove these two lines from the parser:

// Force characters-as-bytes to work.
aCode = aCode.replace(/('(.){1}')/g, "$1.charCodeAt(0)");

With those lines removed, the tests came back green and it seemed that particular beast was on its way into the cage. But you know what they say about things that seems too easy.

Corban was quick to show that we still needed the auto-conversion facility and if something was to be done, removing those two lines was definitely not the end of the story. Within six hours, he'd come up with a much craftier solution: a Char class. Now, functions could check if an argument was of type Char and act accordingly. Problem solved!

#312 Char arguments indistinguishable from numbers

Along the way, Anna poked me to take a look at ticket #360, noting that in this example the yellow letters were appearing white. This was due to the char-to-int conversion issue, and when that was solved, #360 was resolved without any additional work.

#360 basic letters example displays wrong info

All's well that ends well, yes?

A More Recent Note: Release 0.8

The ride on the 0.8 line started innocently enough with the conversion of some visual tests into unit tests. Two failing tests for binary() had to be corrected. One was caused from an incorrect comparison value (Pjs was otherwise returning the right value). The second error was fixed by converting binary() to detect objects of type Char. Along the way boolean() was changed to throw errors on invalid arguments, instead of returning undefined.

There was a short stop at ticket #269, concerning whether or not a cloned ArrayList should contain deep copies of items contained in the original. Processing uses Java's ArrayList, which does not deep copy. Since Pjs' ArrayList already conformed to this, the ticket was resolved as invalid.

#269 ArrayList.clone does not deep copy

A rather longer stop was for printMatrix(). printMatrix() outputs the current transformation matrix to the console. Work started in 0.7 but at the time it only worked when Pjs was working in a 3D context. Why? Because Pjs handles the 3D matrix itself, courtesy of PMatrix3D, and so the values of the 3D transformation matrix were readily accessible. When working in the 2D context, Pjs used canvas' internal matrix. This worked very well, except there was no way to retrieve the values of canvas' transformation matrix, and so there was nothing for printMatrix() to print.

#174 printMatrix()

printMatrix() was ultimately made ready for 0.8, thanks to work on a new and wonderful class...

PMatrix2D

During a phone conference in early March I proposed the creation of PMatrix2D to get printMatrix() going. This meant replicating everything canvas already did just to be able to get the transformation matrix values. I was hoping I was missing something and that there was another, easier way, but apparently there was not. At the end of March the green light was given to cobble together PMatrix2D. Maybe the canvas API will be expanded in the future to allow the values to be retrieved.

Choosing where to start implementing PMatrix2D for release 0.8 was easy: I coded just what was needed to convert the existing PMatrix3D and PMatrix3DStack tests and examples into their 2D equivalents.

Rather than creating a new PMatrix2DStack class, it was easier to transform PMatrix3DStack into PMatrixStack which handled both 2D and 3D matrices.

Toward Release 0.9 and 1.0

Things seem pretty clear cut from here on in. For 0.9 I hope to finish off implementing the rest of PMatrix2D's functions. 1.0 will aim to get Pjs actually using it.

Processing.js – Some bits for 0.6 in time for midterm season

2010-02-15T14:41:00.000-08:00

The Foundations of the Universe are based upon certain Truths, like: as the time to March break/reading week approaches zero, the frequency of exams and due dates sky rocket. Coincidence? There's got to be a formal proof somewhere on the Internet...

But before the coming crunch time gets real bad, is 0.6! From a developer's standpoint, the big improvement for 0.6 is unit testing. Now, just testing return values no longer requires cobbling together a full Processing program, and the tests may be more easily reused and modified by other developers. Yes, this means “pretty pictures" will appear far less frequently now., but on the whole is that really a bad thing? (Using canvas and Processing just to spit out numbers and letters doesn't do justice to either.)

char(), float(), and str()

First up: enhancing a trio of conversion functions. They may now handle the range of primitive type and array arguments they are supposed to.

While the functions themselves don't do anything particularly remarkable, their behaviour helped uncover a bug in Pjs. When user code is parsed, Pjs converts chars into numbers. The relevant code is inside the Processing.parse() function:

// Force characters-as-bytes to work.
aCode = aCode.replace(/('(.){1}')/g, "$1.charCodeAt(0)");

The parser finds single characters flanked by single quotation marks and calls charCodeAt() on those characters. What this means is overloading Pjs functions to handle both arguments of type Number and characters (essentially one-character Strings in JavaScript) becomes... difficult.

Take str('A'). This should return the string “A". But because of the parser what actually gets processed is str(65) (the number 65), returning the string “65". Without jumping additional hoops, you can't tell str() whether that 65 is supposed to be a number 65, or if you actually mean that 65 to be representative of the char 'A'.

This causes some str() tests to fail. char() tests are affected although they all pass. For example:

int i = 65;
_checkEqual('A', char(i));

This is evaluated as _checkEqual(65,65), not _checkEqual('A', 'A'). The parser's fudging is masked by the passing tests.

I've filed a ticket with the suggestion that the parser not convert chars to Numbers, and leave functions to do this by themselves. With unit testing now rolling it'll be easier to find which functions rely on the parser's current behaviour. I may very well pick up the ticket for 0.7.

normal()

normal() takes three numeric arguments representing a vector and saves them to global variables. Oh, and it changes a flag. All of these (variables and flag) are supposedly used by other functions, but none of those uses have been implemented yet. (“What came first, normal() or the code that uses normal()?")

Since normal() has no return value, the unit tests content themselves with checking the correct number and type of arguments are present.

#125 normal()

smooth()/noSmooth()

char(), float(), str(), and normal() are the good news. Alas, smooth() and noSmooth() fall somewhat short of that mark and probably won't appear in 0.6.

smooth() and noSmooth() are intended to toggle anti-aliasing on and off when painting. Canvas has no built in facility for this. sephr found a Mozilla-only CSS property image-rendering applicable to the canvas tag, and perhaps able to achieve what was desired.

Unfortunately, while image-rendering is good for raster images it doesn't work the same magic on canvas drawings. For the time being, it seems smooth() and noSmooth() will go into hibernation.

#145 smooth() and noSmooth()

Processing.js - It's a long way to 0.6

2010-02-07T21:20:00.000-08:00

“But my heart's right there", or so the song goes. Indeed, it looks like my work is laid out for me. So for this short update, I'll just list what I hope to get done for February 19th (in no particular order):

Port normal() (Ticket)
Complete porting smooth() and noSmooth() (Ticket)
Upgrade char(), float(), str() to accept a multitude of argument types (Ticket)

This will also be the first release where we'll be using automated tests! I wonder if I still have Minefield stashed away somewhere...

Processing.js – int() and Release 0.5

2010-02-05T03:13:00.000-08:00

int(), the late passenger

A long time ago, in the forest across the river, where fiery leaves danced upon the crisp winds of pseudo-winter (or in other words: back in the 0.2 release) I finished porting boolean(), a function that converted primitive types to either true or false, over to Processing.js (Pjs). As it transpires, boolean() was just one a gaggle of conversion functions, many which continue to seek the company of a code monkey willing to give them a boost over the wall from Java-land into JavaScript-land.

Yesterday evening, at the behest of The Powers That Be, and against the questionable judgement of my lazy self, I tentatively picked up the rest of these converters. And as I told Anna, the recently promoted Fearless Leader, I'd try to get one of them done for 0.5, whose deadline is today. So, without further ado, I present to you int() for Pjs.

Who wants to write the thousand words?

Lo, see now int() in action in Pjs, and compare it to how the same code renders in Processing:

Your eyes do not deceive you: Pjs' int() fails a test. But don't break out the torches and pitchforks just yet! There's a good reason for this. int() may take a byte or an array of bytes as an argument. In Processing, a byte may have a value from -128 to 127, but more importantly: byte is a static type. Through overloading this allows Processing methods, written in Java, to behave differently to accommodate different variable types.

Things are not so simple in JavaScript. In Pjs a Processing byte and int gets converted into a JavaScript 'number'; effectively there is no distinction between a byte and an int in Pjs. Since there's no distinction, int() cannot be tailored to behave differently when it receives bytes. (floats and doubles are different since you can always check if there's a decimal point in there somewhere.) Therefore, the test fails.

A possible solution may be to add a second parameter to int(), a flag called 'isByte' or something. That might be something to bring up in next week's telephone conference.

int() links

A Quick Recap of 0.5
Just for the record, here are the links to the rest of the stuff I did for 0.5:

I'll be making a (hopefully) shorter post soonish outlining what I hope to accomplish for 0.6. But for now, it's time to switch gears and call it a morning.

Processing.js – nf()

2010-02-02T19:23:00.000-08:00

Closer to release 0.5

With a little luck, this should be my the last piece for the 0.5 release of Processing.js (Pjs) . The nf() function should now be able to handle arrays of numbers to be formatted, and float formatting.

There's a few interesting things going on here that the Processing documentation doesn't allude to, but those are more easily revealed once we see the code in action.

A few pixels

Here's the first batch of tests in Pjs (hah, there's no way I'm trying to minify that for the post!) and here's what Processing generates for the same code:

And here's the second batch of tests (arrays) in Pjs, and here's what Processing generates for the same code:

Note the tests use String::equals() which I recently completed and is not yet in the public release.

Also, while the documentation says all arguments should be positive integers, Processing's nf() does handle arguments with values zero or less. So I've coded Pjs' version to do the same.

It's supposed to do that? behaviour.

There are two weirder-than-normal things going on.

In the tests 19 to 32 in the first batch you'll find that depending on whether the one's digit is odd or even, the 'rounding' is different. This required a bit of extra code to replicate.

In the second batch, nf() processes values in the int array “incorrectly"; running through the ints individually results in the expected results. Strangely enough, my code was able to replicate this behaviour right off the bat.http://wobblyretroaction.blogspot.com/2010/01/processingjs-stringequals-or-how-i.html

Links

Processing.js – String::equals() (or: how I filled something in on the way)

2010-01-31T19:18:00.000-08:00

Pothole? What pothole?

Open source development is like driving on a road with potholes where every driver also happens to be a road worker. One day you may just get fed up with driving over the same crater over and over again and decide to do something about it in the most direct way possible: stop the car, get out, and fill it in yourself.

Of course, some holes get away for a while because they're just not that bad, or too deep. Other's scream for immediate rectification. Like the maws that open up suddenly and force you to slam on the breaks less you go careening into the deeps of the Earth. Or the holes just big enough to swallow a wheel and bring the scheduled excursion to a halt.

Well, on the way to giving nf() a nudge I discovered one of those wheel eaters. Like Java, Processing uses an equals() method to compare two Strings. JavaScript gets away with just ==, and for the past while Processing.js (Pjs) has done without equals() and used just ==. This was all well and good... until I started writing tests for nf().

The nf() tests compare the String result of nf() and an expected String result. Depending or not the two Strings are the same the output's colour changes. To ensure the code used for the test in the Processing Development Environment and in Pjs were the same I needed equals(). (Mind you, I could have used == in Pjs, equals() in Processing, and get the same output, but that would have been cheating.)

So with that short story aside, I present a quick and dirty String::equals() for Pjs. May this hole never become deeper.

How the road looks now

So here's how it looks in Pjs. (Again, we'll have to make due with a link. Minifying the code for insertion into the post didn't work out to well.) This example is extended from the Processing example.

And here's how the same code looks like in Processing:

Links

Processing.js – Improving min() and max()

2010-01-31T03:48:00.000-08:00

And another brick

So here's another little improvement for Processing.js.

In Processing, the min() and max() functions may take 2 or 3 numeric arguments, or one argument if its an array of numbers. For a while the Pjs implementation could only take two arguments. I have since taken it in hand and the Pjs implementation should now be able to do what the Processing implementation can do.

Well, not exactly. In fact, the Pjs implementation will happily take a great many more numeric arguments than just three. It'll even take one number in a pinch. Passing in non-numeric arguments, aside from a single array, will cause "undefined" to be returned. Trying to be clever and pass in an array as one of two or more arguments will also return "undefined". In Processing these "undefines" would be "thrown exceptions".

My eyes!

This will be another post where I'll provide links to the Pjs tests and comparison Processing images.

Basic Processing example for min() in Pjs, and how it looks in Processing:

Basic Processing example for max() Pjs, and how it looks in Processing:

3 and 4 numeric arguments, and argument type discrimination, for min() in Pjs, and how it looks in Processing. Note that only three arguments is tested in Processing since the rest cause exceptions.

3 and 4 numeric arguments, and argument type discrimination, for max() in Pjs, and how it looks in Processing. Note that only three arguments is tested in Processing since the rest cause exceptions.

For the tests I had to pull a patch or two fixing Ticket #57 Boolean array bug.

Links

DPS911 Project – Processing.js – v0.4

2010-01-20T05:50:00.000-08:00

A bit of touching up
So here we are: the first release for DPS911 this semester! For this release I've added missing functionality to Processing.js's (Pjs) implementation of text(). Unlike before, the function may now handle the printing of variable of integers, floats, doubles, and bytes, in addition to strings.

Now I wait for a (hopefully) favourable peer review.

Pretty pictures
Here we see a live demo of how it works in Pjs now as shown on this page:

And here we see the same demo as displayed in the Processing Development Environment:

What are we looking at?
The Pjs implementation had to take a few things into account. Processing displays floats and doubles to the thousandths digit, which is easy enough to replicate in JavaScript using toFixed(). But notice how Processing rounds: for 2.7185 it rounds down to 2.718 instead of up to 2.719.

toFixed(), by itself, rounds 2.7185 to 2.719, so a little bit of extra code was needed to alter this, like so:


if ( (val*1000) - Math.floor( val * 1000 ) == 0.5 ) {
  val = val - 0.0001;
}
val = val.toFixed(3);

So for 2.7185 we craftily modify it to 2.7184 before sending it to toFixed().

Pertinent links

Thoughts on the Proper Care and Feeding of Git Repos on GitHub

2010-01-17T11:24:00.000-08:00

The nature of the problem

Last semester I and the other students working on Processing.js (Pjs) started using GitHub. We all made our own branch of the Pjs trunk repo and started hacking away. Over time we committed to our branches and some of our improvements went into trunk.

But our very success started to have some unintended consequences. Trunk advanced but our branches did not gain the improvements merged into trunk. This made our antiquated branches increasingly poor bases to produce patches for trunk. Needless to say, this inconvenience was not appreciated by anybody.

After a bit of experimentation I managed to find a way to keep my branch up to date without deleting it and starting fresh. Mickael Medel (aSydik), one of my classmates, has requested I spread the joy and blog about what I did. So here it is.

Fractional-reserve coding

In five easy steps most, if not all, problems, may be averted.

Step 1 – Pull from trunk to the branch on your local system

Before making any changes locally make sure you have the latest version of trunk. Pulling from trunk every time it changes isn't such a bad idea: it's much easier for Git to automatically merge code when the difference is small or simple. If the difference is too complex Git may cry "conflict!" and make you sort out the mess. Not good.

For us monkeys working on Pjs, I believe the pertinent pull command is:

git pull git://github.com/jeresig/processing-js.git master

Now, if you have more than one development system (as I do: desktop at home, and laptop) you may need to pull from your own repo instead of trunk. In that case the pertinent command is something like:

git pull git@github.com:yourusername/processing-js.git master

Step 2 – Hack on your local branch

So now have a totally updated version on your local system and the time has come for you to work your magic. Hack, surf, and make merry!

Step 3 – Commit on your local branch

After many hours of fruitful hacking you look out the window and the sun has been suspiciously replaced by the moon. It's time to call it a morning and push things back to your GitHub repo. But wait! Remember to commit the changes first, otherwise you'll have nothing to push.

I guess that Step 2 and 3 are just reminders, right?

Step 4 – Push your local commits to your online repo

Now comes the culmination of your efforts, fuelled by countless grams of caffeine and other tasty goodies: the big push. At this point you need to update your online repo to match the local repo. Not to hard, just plug in something that looks like this:

git push git+ssh://git@github.com/yourusername/processing-js.git

If all goes well the local commits will flood across cyberspace, and you shall know balance and harmony...

Step 5 – Back to Step 1

… for a little while. When you're ready to go at it again, collect your $200 and sally back to Step 1.

Opposites attract, but sometimes we wish they wouldn't

The above works very well if your branch is kept up to date. But if you're sitting on some really old revision chances are Git will be unable to do the merge in Step 1. It'll cry "conflict!" and you'll have to sort it out. By this I mean you'll need to do the merge manually (yes, lots of copying and pasting and what not.)

If the situation is really bad you may simply want to kill and remake your branch from scratch, keeping in mind to regularly update the resuscitated branch this time round.

If you slog through the manual merge, then commit changes to finish the merge. Committing will also tell Git the conflict has been resolved and you'll be back in business. At least, that's what I think happened when I resolved my first (and so far only) conflict.

Only now, at the end, do you understand.

I hope that was helpful for those in need.

I apologize to Mickael for the tardy post. He asked for this last Thursday and it took me a while to cobble this together. You see, I use an unholy combination of the command line and the EGit plugin for Eclipse to interface with GitHub so I wasn't quite sure what the exact commands were. I tested them with my latest commit to GitHub so I hope they work for everybody else.

Yes, I use Eclipse to hack Processing.js, but only because EGit appeals to my laziness. Otherwise I'd do what sane people do and use a "regular" text editor.

2010, DPS911, and more Processing.js!

2010-01-14T01:05:00.000-08:00

The Times They Are a-Changin'
Last semester in 2009 (so long!) I wandered into DPS909 and busied myself porting some functions over from Processing to Processing.js (Pjs).
For more on what Pjs is about check out a previous ancient post.
In a fitting continuity of events , this semester in 2010 (hello!) I've wandered into DPS909's follow-on, DPS911 and will continue futzing around with Pjs. The work done for DPS909 were released as 0.1 to 0.3, so not surprisingly the work for DPS911 will be released as 0.4 to the big 1.0.

The Plan
For v0.4 to v1.0, I hope to port over some new functions in addition to fixing existing bugs. The prospective task list and timeline is given below:

Release 0.4 – Jan. 22, 2010
- Fix text() to accept non-String arguments.
Release 0.5 – Feb. 5, 2010
- Fix min() and max() to take more than two arguments each.
Release 0.6 – Feb. 19, 2010
- Port loadBytes().
Release 0.7 – Mar. 13, 2010
- Port normal().
Release 0.8 – Mar. 27, 2010
- Port printMatrix().
Release 0.9 – Apr. 9, 2010
- Port printCamera().
Release 1.0 – Apr. 23, 2010
- Padding for overruns and to give time to fix any bugs from releases 0.1 to 0.3 from DPS909.
- If that happens not to be necessary, I'll just choose another function to port of bug to fix.

Obligatory List of Relevant Links

DPS909 Project – Processing.js – v0.3

2009-12-09T16:23:00.000-08:00

And It's A Warp...
… for this phase. This is the last release for this semester's DPS909 project. For this bit strokeCap() and strokeJoin() were ported from Processing to Processing.js. The meat of what was done may be found in previous posts linked to above, so I won't occupy everybody again by repeating it here (after all, it's exam time!)
So without further ado, here are pertinent the pertinent links:

Links to Elsewhere

Lighthouse ticket for strokeCap()

Lighthouse ticket for strokeJoin()

Code branch at GtiHub

Links to the Past

Release 0.1

Release 0.2

And Away!
That's it for now. With a little luck I'll see you right back here in a months time for DPS911.

Processing.js – strokeJoin()

2009-12-02T12:48:00.000-08:00

Copy and Pasting,,, Works!
Hot on the heels of strokeCap() strokeCap() is strokeJoin(). This blazingly fast turn around was accomplished by the time honoured method of shamelessly lifting previous work wholesale and tweaking.
Observe: here we have strokeCap():

p.strokeCap = function strokeCap( set ){
        if ( set == p.ROUND ) {
                curContext.lineCap = 'round';
        } else if ( set == p.SQUARE ) {
                curContext.lineCap = 'butt';
        } else if ( set == p.PROJECT ) {
                curContext.lineCap = 'square';
        }
    };

And here we have strokeJoin():

p.strokeJoin = function strokeJoin( set ){
     if ( set == p.MITER ) {
      curContext.lineJoin = 'miter';
     } else if ( set == p.BEVEL ) {
      curContext.lineJoin = 'bevel';
     } else if ( set == p.ROUND ) {
      curContext.lineJoin = 'round';
     }
    };

As we can see strokeCap() gained a very close sibling in strokeJoin()!

Newborn Boo-boos
This is not to say the birth of the younger didn't affect the elder. Originally the constants for strokeCap() and strokeJoin() were defined separately, like so:

// Stroke cap constants.
     p.ROUND   = 0;
     p.SQUARE  = 1;
     p.PROJECT = 2;

     // Stroke join constants.
     p.MITER = 0;
     p.BEVEL = 1;
     p.ROUND = 2;

p.ROUND was redefined for strokeJoin() and ended up conflicting with strokeCap()'s p.PROJECT. This caused... incorrect behaviour in an otherwise well behaved strokeCap().
The problem was solved simply be consolidating the constants into:

// Stroke cap and join constants.
    p.ROUND   = 0; // Used by both cap and join.
    p.SQUARE  = 1; // Used by cap.
    p.PROJECT = 2; // Used by cap.
    p.MITER   = 3; // Used by join.
    p.BEVEL   = 4; // Used by join.

Pretty Pixels!
To end off here is the code in action compared to the results of the same code in the Processing Development Environment (PDE).

On a side note I've noticed the Processing website's examples are imprecise; their code and images do not match up. The images show the result of strokeCap(SQUARE) or strokeCap(PROJECT) being called, but these calls are missing in the example code.

Processing.js – strokeCap()

2009-11-29T17:03:00.000-08:00

strokeCap()
Another week or two, and a bit of work toward the 0.3 release of my DPS909 project. This time it was strokeCap()'s turn to be ported to Processing.js (Pjs).
So here is the basic Processing example working in Pjs:

And here is the example running in the Processing Development Environment (PDE):

Squishy Innards
This is all done by setting canvas' lineCap attribute. lineCap has three possible values which correspond directly to those used by Processing so strokeCap()'s code looks just like this:

p.strokeCap = function strokeCap( set ){
    if ( set == p.ROUND ) {
        curContext.lineCap = 'round';
    } else if ( set == p.SQUARE ) {
        curContext.lineCap = 'butt';
    } else if ( set == p.PROJECT ) {
        curContext.lineCap = 'square';
    }
};

lineCap is also handled by canvas' state saving functions, which means Pjs' pushStyle() and popStyle() can already handle saving and restoring the setting as well.

A Different Starting Point
The main difference between canvas and Processing/Pjs is the default setting for lineCap. Canvas' default is “butt", which is “SQUARE" in Processing. Processing needs to default to “ROUND", which is “round" in canvas.

There was a bit of trouble getting the default working in Pjs. At first adding curContext.lineCap = 'round'; to function init didn't work. Al MacDonald (F1LT3R) was very quickly able to find where the problem was and fix it: it turns out function size recreated the canvas context every time it was called, which in turn reset all the attributes. Thanks, Al!

So now the default setting works very nicely in Pjs:

Compare to the same thing in PDE:

Useful Links

GitHub commit where strokeCap() works

Lighthouse ticket (will need to be reopened for code review)

Firefox Addon and XPCOM

2009-11-26T09:52:00.000-08:00

So last week in DPS909 we were supposed to feel our way through a lab giving some idea how Mozilla's XPCOM works. XPCOM is used to ensure code portability. I do not pretend to understand how it all works, it's rather complicated and terrifying for a newbie, but at the very least I can follow instructions.

The big first step is to create an XPCOM interface using a XPIDL file, which includes function prototypes using familiar C/C++ data types (at least in the walkthrough, XPCOM may be used with other languages to). A header file is automatically generated which, amongst other things, transforms the data types into Mozilla's data types and creates a stub class using the header. In other words, a lot of the basic set up work is done for the code cruncher (winner!)

Most of the walkthrough was pretty straight forward. There were a few minor things that needed to be changed or were different. In install.rdf, I changed em:minVersion and em:maxVersion to 3.5.5 and 3.7a1pre respectively (that allowed the extension to work in the trunk build.) Also, IFirstXpcom,h was created in $(objdir)/dist/include/ rather than $(objdir)/dist/include/firstxpcom/.

But everything worked out and so I can now claim to have created (after near slavishly following somebody else's instructions) an add-on for Firefox.

Processing.js – Something Lost/Gained in the Translation

2009-11-20T18:00:00.000-08:00

The nice thing about porting is the end result is already known. Comparing the effects of the original with the effects of the transcription is a good indicator of how close you are and whether or not you're on the right track.

Simple, yes?

Sure, at least in a perfect world where I could have my kitchen faucet yield hot chocolate and have my computer magically upgrade itself to the latest hardware and software. Alas, a perfect, honest-to-the-source, zero deviation transcription may not always be possible; such is what I have discovered while tinkering with pushStyle() popStyle() and boolean().

Below you will find I have not embedded the Pjs examples, instead I shall provide a link to another web page with the example on it. This is because embedding requires minification, and I'd rather not go through the frustration of minifying some of the larger examples. I'll provide screenshots of how the examples look using the Processing Development Environment (PDE) for comparison.

Differences in pushStyle() and popStyle()
The basic Processing reference example looks like this

and can now be reproduced in Pjs (Pjs demo here.)

But take a Pjs demo like this one. See the extra pops at the bottom? You can't do that in Processing. The PDE won't allow the Pjs example code to run; it'll see the extra pops, throw an exception, and refuse to run properly until the extra pops are removed. The closest approximation to the Pjs example Processing will allow looks something like this:

Pjs doesn't really concern itself with catching logical errors in the code. At best it just concerns itself with syntactic errors, and if there are any it simply doesn't paint anything onto the canvas. Saying that, the Pjs version of popStyle() has to be a bit more lax than its Processing counterpart and work with, rather than crash on, extra pop calls.

In this matter the solution is simple. Upon figuring out the pop call is extraneous, Pjs effectively ignores it and carries on. It must be one of the few times burying one's head in the sand actually makes the problem go away!
Differences in boolean()
Moving onto boolean(). Let's start out with showing a basic example based on the one found on the Processing website. It looks like this:

and it too may be reproduced in Pjs (Pjs demo here.) However, boolean() is afar more complicated beast than the basic example would imply. In this case it's a matter of how Java/Processing is very strict on the number and nature of arguments passed to a method, while JavaScript/Pjs is not.

Take for example the following this Pjs demo. Note how we're passing in doubles, booleans, ints, nulls, undefines, etc., and arrays with mysteriously empty elements. If you were to try the code in Processing it wouldn't work. For one, Processing's boolean() refuses to handle boolean variables, or double variables, or nulls, and other fancy things like that. In fact, the closest Processing example to the Pjs demo is rather more sparse and looks like this:

(Un)fortunately, Pjs' boolean() can't take the easy way out and throw away arguments it doesn't like, coughing up an exception, and making the whole show grind to a halt. Being JavaScript it's syntactically correct to pass a cornucopia of dynamically typed arguments into a function. Pjs' boolean() has to handle whatever comes its way gracefully.

As you can tell from the Pjs boolean() demo, I thought this was best done by having the function return "meaningful" results from arguments not normally handled by Processing's boolean(). Numeric arguments are handled like integers: zero values return false, non-zero values return true. Boolean arguments are just returned as themselves. Nulls and undefines return false.

Heresy!
I like to think I've cobbled together an acceptable compromise between how Pjs and Processing behave. I guess I'll find out soon whether this is acceptable or not when I try to push the code into the library.

DPS909 Project – Processing.js – v0.2

2009-11-19T21:06:00.000-08:00

Returning to Ground Zero
First, a self-plagiarized recap. This semester at Seneca College I am taking a course revolving around open source development. As part of the course the students (yours truly being no exception) were nudged into getting involved in some Mozilla related projects, produce code for the real world, and see fist hand how the whole open source thing works.

I chose to work on Processing.js (Pjs), a JavaScript port of the Java-based Processing. More specifically, I volunteered to work on porting over the functions pushStyle(), popStyle(), and boolean().

On 19 October 2009, I threw release 0.1 out into the cruel world to fend for itself. Today I present release 0.2 to the world. Applause!

Not new, but shinier!
The watchwords for release 0.2 were: fix it. Release 0.1 included reasonably functional code for pushStyle() and popStyle(), and totally untested and untried code for boolean().

Bringing boolean() up-to-spec was the goal of this release and this was done by throwing together some test cases for boolean(). Bugs were indeed caught, squashed, and flushed down the toilet.

Although boolean() got most of the attention, the elder siblings did not totally escape notice. pushStyle() and popStyle() received their own test cases too. One issue was whether the pop should spit out alert popups if there were not enough pushes. I decided not to use any popups and just have a warning be sent to the console, where most users probably won't notice. pushStyle() and popStyle() were further improved to better handle the library's own variables; before they only managed canvas' variables with any proficiency.

Better to get it when it's pleasantly warm rather than skin-charring hot

Download the full Pjs library (my patched version) OR

just download the patch

What now?
Pursuant to getting my code merged into the Pjs library there are a few instructions I have to follow. So I'll make another post comparing the results of my Pjs code with those of the Java-based Processing code. As I have discovered it is not totally possible to faithfully port Processing to JavaScript. For example, Processing will throw exceptions and refuse to do anything if you try to pass an argument to a function and the function is not designed to accept, say, the variable type of the argument. But in Pjs you have to handle these weird arguments and preferably return some meaningful result. This comes up a few times in boolean(). But more on that soon.

For release 0.3, it looks like I'll have to choose another function to port over, or at least get started on even if I don't finish by the end of the semester. strokeCap() looks appropriately involving.

Learning a Bit of JavaScript

2009-11-07T21:23:00.000-08:00

This week in DPS909 we talked a bit about JavaScript. This was particularly relevant to my project this semester, implementing a few functions in the Processing.js JavaScript library. As part of this week's journey of the mind, we were directed to watch a lecture by Douglas Crockford called "JavaScript: The Good Parts". Leaping over to Wikipedia for some familiarization, Crockford is the senior JavaScript Architect at Yahoo!, and wrote the specification for JavaScript Object Notation. The lecture in the video is based off his book of the same name.

Right off the bat, I'm one of those people who, as Crockford' puts it, started tinkering with JavaScript without learning it. So given my novice credentials coupled with a pressing need to "learn more" I'm pleased that the lecture had a few "do and don't" pointers that someone like me could take away.

The first big thing I learned was use ===, avoid ==. I was first exposed to === by David Humphrey, who gave me a code snippet demonstrating how to detect if a JavaScript variable was an array. A bit of research revealed === compares not only the value of two variables but their types as well, a very useful feature in a language with dynamic typing. I was, not, however, inclined to use === near exclusively.

After watching Crockford's lecture made me realize using mainly == would eventually cause grief.. Sooner or later I would make compare two values, forget the variable types, or forget the distinction between null and undefined, and end up with an annoying, very small, difficult to track down, bug. Using === would be an easy way to save a bit of heartache in the future, which is always a good thing.

Learning the distinction between null and undefined was an ancillary to the first major thing I picked up. The former is a value for an object variable, the latter can be had if trying to retrieve a variable an object doesn't have. Note to self: further research required.

The second, and most surprising, thing I learned was coding style matters. It's easy to mistake JavaScript for a free-wheeling language: dynamic typing, you don't have to use semi-colons half the time! But apparently something as "simple" as bracket placement when defining blocks can really mess things up. Crockford gave the following example:


return 
{
 ok: false
};

This seems innocent enough, but apparently behind-the-scenes shenanigans means it actually is interpreted like so:


return;
{
 ok: false;
}

In other words, everything after return goes nowhere and does nothing. For this to work as intended the opening bracket must be placed to the right of return like so:


return {
 ok: false
};

While my coding style generally resembles the "correct" example, this will be a useful tidbit when deciphering the code of other programmers.

Fortunately, it seems Crockford has developed JavaScript code quality checker called JSLint to help developers catch some of their more egregious code. Methinks I shall be availing myself of it in the near future.

Standing On Their Shoulders: A Short Processing.js Tale

2009-10-31T16:31:00.000-07:00

So this semester's reading week has come and gone and once again, as is the familiar post-mortem assessment, I have discovered the ratio of “useful" to “superfluous" work done is heavily biased toward the latter.

On the Processing.js (Pjs) front we have more baby steps!

Don't mind if I do...

For pushStyle() and popStyle() I've come full circle. First I fiddled around with storing and retrieving variables in the library, then I moved onto storing and retrieving canvas variables, and have since returned to working on the Pjs library variables.

In my previous post, I discovered some code by Andrew Peace. As it turns out, not only did Peace's work handle the Pjs variables, he also did it in a more JavaScript savvy way then I would have done.

The example that sticks out in my mind was his handling of arrays. While I was busy incrementing and decrementing a counter to figure out how many states I had and which element to store and retrieve from, Peace simply used the push() and pop() functions that grace all JavaScript arrays.

I have incorporated Peace's code with only cosmetic changes, and JavaScript's push() and pop() functions for arrays has since entered my corpus of All Knowledge. My hat off to Mr. Peace!

But does it work?

In my previous post I also intimated that some work was going on with boolean(). There was some code but no testing. In the last little bit I've started cobbling together a few test cases, and even thought still incomplete they've already helped flush out a few bugs.

The nice test page layout was provided by classmate Daniel Hodgin. Thanks!

DPS909 Project - Processing.js – v0.1

2009-10-19T18:19:00.000-07:00

Last month on [insert name of media production here]...

First, a brief recap. This semester at Seneca College I am taking a course revolving around open source development. As part of the course the students (yours truly being no exception) were nudged into getting involved in some Mozilla related projects, produce code for the real world, and see fist hand how the whole open source thing works.

I chose to work on Processing.js (Pjs), a JavaScript port of the Java-based Processing. More specifically, I volunteered to work on porting over the functions pushStyle(), popStyle(), and boolean().

I've got to release something...!

Today I get to make my first (course mandated) release! And this is what it does (trotting out this little “demo" for the third time):

What's that? That, fine feathered friends, is pushStyle() and popStyle() working just a little bit. Nothing particularly amazing, it's just the Processing example copied verbatim.

A start on the boolean() code is also in the release, but it's not tested at all.

For those of you who want to see for yourself may:

Download the full Pjs library (my patched version)

Download just the patch

Toward 0.2

The march to enlightenment continues, and there is still much to do. The implementation of pushStyle() and popStyle() remains incomplete as demonstrated by this discussion on the Pjs Google Group. So far I have only saved the state of canvas variables, and not those in the library itself.

Then I need to see about putting through a more expansive test regimen. And there's already a bug: using Andrew Peace's test case, the “more pops than pushes" error alert just keep popping up. I like to think the intended behaviour is for a finite number of warnings.

On the other hand, boolean() seems much more straight forward to test.

A not-so-secret society

v0.1 releases just in time for this years Free Software and Open Source Symposium hosted by Seneca College. This is particularly important because Al MacDonald, Pjs project leader, is attending FSOSS and will no doubt want to have a few words with those DPS909 students working on Pjs.

David Humphrey suggested us students might want to look into setting up a Pjs bug tracker, discuss ways to compile our seperate test suites into one big suite, and perhaps set up a website to showcase the newly ported capabilities. I figure even if I somehow manage to get my project done way ahead of schedule there will not be a lack of things to do.

Fixing a Bug In A Harmless Test Environment

2009-10-17T13:22:00.001-07:00

This week's DPS909 lab brought together everything covered thus far in the course: build the software, file a bug, find the offending code, make a patch. If this were any other course this lab might qualify as a "quest": not a quiz, not a test, but somewhere delightfully in between!

Step one was compiling a debug build of Thunderbird/Shredder. I checked out straight from the repo rather than use the code provided by the lab instructions. The process was similar to building Firefox/Minefield and went off literally without a hitch. A very palpable hit for me!

Nest was to confirm the bug's existence. The bug we were interested in deals with Thunderbird converting a string that looks like an email address into a hyperlink; the problem is the string does not represent a valid email address and so should not be converted. We can see the fubared behaviour below:

So having confirmed the existence of the bug without difficulty, bug #7243 was filed in the landfill. I like to think I included the bare-bones information in the description answering the questions of: "what is wrong?", "how may it be reproduced?", and "what version of the software are you using?".

Now came the matter of tracking down that needing futzing with. This took a while. I spent much time plopping in search terms into the Mozilla Cross-Reference without success. I looked into "parse message", "save draft", and the like to try and get into where the message was stored and perhaps displayed to the screen.

In desperation I went back to the program and, mousing over the offending email link, saw it was a mailto link. So I through in mailto as the search term and... there was light. (Just goes to show simplicity is golden.) Conveniently the very first result from that search took me exactly where I needed to go.

if (inString.FindChar('.', pos) != kNotFound)

The conditional only checked if a "." was found after the @ sign. What it also needed to do was check if there were any ".." after the @ too, which would mean it was not a valid address. Some way needed to be found to find whether a ".." was in the string. Unlike what was currently in mozTXTToHTMLConv.cpp I needed a way to check the presence of a string in the string, and not just a char in the string. According to the Mozilla internal string guide the correct way to find if a substring existed was to useFindInReadable in nsScannerString.cpp, However converting the code to use the newer approved way was probably a bit beyond what I wanted to delve into for this lab.

So I just stuck with the old way. Searching through the documentation some more I found nsDependentString had just the function I needed. The code mozTXTToHTMLConv changed to:

if (inString.FindChar('.', pos) != kNotFound && inString.Find("..", pos) == kNotFound)

A recompile later the situation was much improved:

I created the patch from ./comm-central/mozilla and ran it through JST Review. A few "longer than 80 characters" and extra whitespace/tabs warnings later, it was ready to be attached to the bug report.

Ehren graciously took some time to review the patch and found I had indented a line a bit too far. It was up to a patched patch to pass muster instead.

The tool-tips for the review options for attachments threw me off a bit. Super-reviews are only needed for world-shaking changes, but the super-review tool-tip says something somewhat different. I didn't request a super-review for the revised patch.

The patch and review comments have been copied to my Seneca College wiki page.

boolean() for Processing.js

2009-10-12T19:08:00.000-07:00

And so we return for another (short) post in the irregular series on “What is Matthew doing with Processing.js?" Ah, the joys of lagging a bit behind from where where I should be. But onwards to putting a little bit more on my plate!

Seeking truth From variables

There is a spectre haunting the incompleteness of Processing.js (Pjs) (well, one of many spectres) - the spectre of boolean(). It is a spectre I hope to, at least, partially exorcise by December.

This little function takes a variable, may it be integer, String, object, or an array thereof, and spits out “true" of “false" (or even an array of such). A little peeking into the original Java code shows that the function is actually spread over five overloaded Java methods in class PApplet.

Of course, it will all have to be condensed into a single function in Pjs. Since it doesn't seem JavaScript requires the explicit declaration of data types, I'll have to search around to see how one might go about distinguishing between them. However I expect there will be plenty of examples to sink my teeth into. (I can't be the first person to ask: “How do I tell if a variable is an array?" after all!)

The project page has been given a minor revision to reflect the new bit on the plate. I really must get around to jotting down some documentation on my latest work with pushStyle() and popStyle(). See? Lagging behind.

Speaking of which...

In my previous post I displayed a basic working example for pushStyle() and popStyle(). Alas, I hosted the JavaScript library on a temperamental web host so it can't be seen half the time.

I redisplay the example here using (hopefully) a more reliable web host.

But where has the time gone? Release version 0.1 is just around the corner, and there is still things to ponder and code to crunch! I guess it would not be a student's life if one did not feel kinship to the tardy White Rabbit.

According to the Plan

2009-10-03T15:07:00.000-07:00

… this shouldn't be happening.

In a previous post outlining my DPS909 Processing.js (Pjs) project plan, I sketched out the implementation of pushStyle() and popStyle() over a period of over two months. As it turns out this may have been overly pessimistic.

Climbing the Stairs

In the beginning, the scheme called for storing and retrieving states from a nice 2D array. I opted for an object to store each state snapshot.I peeked into the Java Processing code (PStyle.java) to see what sort of variables I might be looking for. The variables listed under the comment // “Private" variables used to maintain state seemed like a good place to start.

Throwing the code into a test page, let's just say those were not, or not all, the variables I was looking for.

Finding the Elevator

Closer examination revealed a mysterious object stored as curContext, and it seemed the object's squishy innards contained what I was looking for. Since those innards didn't seem to be defined in the Pjs library I turned to Google, and that turned up the canvas element documentation.

As it turns out canvas comes fully equipped with functions to save and retrieve state snapshots. So using those functions instead of the array, and state-saving objects, we get something like:

Assuming one's browser comes with the requisite bells and whistles, it should look suspiciously similar to the Processing example.

Expanding the Catch

In terms of getting pushStyle() and popStyle() to work, there are a few things I still need to look into. In my current revision I am not handling any of the library's own variables at all. The basic example works without them but, but some may be relevant state variables and I have to figure out which ones (if any) are so.

Another concern is browser compatibility. Canvas implementation seems to be all across the board right now. If Pjs has to work now, without reservations, then I'll have to go back to the state-saving object and array scheme, while saving all of the canvas variables manually. On the other hand, if the onus is not on Pjs to work on browser's with incomplete canvas implementations, then that's one less thing to worry about.

But the bottom line is that the project as I envisioned it will most likely be done long before early December. This is not acceptable for DPS909 and therefore I must find additional bits of Pjs to hack away at. I'll be figuring that out soonish and changing altering my wiki project page to reflect that.

My Kingdom for a Patch?

2009-09-28T15:21:00.000-07:00

This blog post is brought to us by the letter P and the number 6, and today's topic is “patching". “Patching". (Alas the written word doesn't do justice to the beginning of each Possum Lodge Word Game.)
Patches are funny “little" files that contain the list of changes or differences between two snapshots of a code base. This is good for keeping track of changes to the code, facilitating reversions, and making it easier to spread the joy of testing and breaking systems to self-appointed guinea pigs.

For this DPS909 lab, the first thing to do was hunt for a patch for my Minefield build. I headed over to the Firefox Builds forum at mozillaZine. From Peter(6)'s seemingly endless series on the rise and fall of the nightly trunk builds, I picked out recently resolved bug 424517.

The gist of 424517 was the “use small icons" option was affecting the additional icons added to the Bookmarks Toolbar, when such icons should be start small and remain small no matter what.
I verified that my Minefield and Iceweasel (just because I could) were exhibiting the unfortunate symptoms of the issue. The “refresh" icon in the Bookmarks Toolbar did indeed resize as the “use small icons" option was toggled back and forth.

The patch added a mere two lines, but the important thing was the results could be easily verified. So I applied it:

mlam@mlamprimary:~/src/mozilla-central$ patch -p1 < lockiconsize.diff
patching file browser/base/content/browser.xul

After some rebuilding I prodded open my improved version of Minefield and, lo and behold, it was cured! The “use small icons" had no effect on the Bookmarks Toolbar, just as the Givers Of Data deemed it should be (and there was much rejoicing.) But just to be perverse:

mlam@mlamprimary:~/src/mozilla-central$ patch -R -p1 < lockiconsize.diff
patching file browser/base/content/browser.xul

And after another bit of rebuilding the issue returned as mysteriously as it had vanished. Very sad.

Processing.js – A Plan to Push and Pop Styles

2009-09-27T00:07:00.000-07:00

1 What this is all about

Processing is a Java-based programming language used to display rich graphical content, like animations, on web pages. The content was embedded on web pages using Java applets. Since the time of the original implementation improvements in web technology, such as the capability of web browsers and JavaScript execution speed, has opened the possibility of accomplishing Processing's objective without Java.

Processing.js (Pjs) is a JavaScript port of Processing currently in development. Users need only a sufficiently capable web browser to view the content; there is no need for Java or Flash, their plugins, and the overhead and possible software troubles arising from using those plugins. Matters are also simplified for developers who only need Pjs's JavaScript library.

A convenient collection of links to mailing lists and code repository may be found here. A specific list of "things needing code monkey labour" may be found on MozillaWiki

2 Choosing the road frequently travelled

For DPS909 I and my fellow code monkeys-in-training were directed to jump into a Mozilla-related project. As the old (and perhaps overused) saying goes: I hear and I forget, I see and I remember, I do and I understand. Appropriately for a course about open source development, we were going to do it, and, hopefully, emerge from the experience with greater understanding.

One available set of projects fell under Pjs. Within that project I chose to work on porting pushStyle() and popStyle() from Processing to Pjs.

Before going on I should disclose that I like Java. It's the first object-oriented language I picked up. At the very least, reading the code to be ported would be easier. However, Java was not the major reason for my project choice; that language plays a relatively diminutive part in the overall task. The major part is reading and writing JavaScript which. I have very little experience with JavaScript and working on Pjs presented an opportunity to rectify this for fun and profit! Oh, and there would be ancillary social benefits such as being useful in the process.

Aren't win-win situations wonderful?

I chose tasks from the "miscellaneous feature" list rather than the "3D features" list because the latter constitutes a "here be dragons" area for me. When the course concludes and my work is evaluated, I would rather submit better material and feel silly for choosing something that did not wander far from my starting experience, than submit sub-par material and experience the self-recrimination of having barrelled toward the "bridge too far".

Discretion is the better part of valour.

3 The information must flow

So that a record of progress, U-turns, and backtracking, may be preserved for future generations to point and snicker at, I will be maintaining a wiki page. There is also a branch of the development tree. Hopefully there will be weekly posts on this blog concerning the project work.

4 If everything is going according to plan, it's a trap

As per the requirements for DPS909 the aim is for three "point releases" within the next two and a half months. This means version 0.3 will be available in early December. Per the expectations of the course it is not intended to be the final product although one hopes it will be of sufficient quality for somebody to pick up where I leave off, providing I do not take DPS911 next semester.

The following is a brief overview of the objectives and predicted issues for the point releases.

4.1 Version 0.1 (October 19)

4.1.1 What it to be done

Add a 2D array of style variables with each "row" representing a style set.
Add a variable tracking the style set currently in use. This will be technically superfluous during the interim since only a single style set will be used.
See if we need functions to save the current style variable values or load a set into the variables. Might be overkill.

4.1.2 Gathering the pieces

Install JavaScript Development Tools for Eclipse
Try Egit with github.
Check out Pjs using Egit.
Check out Processing code for reference.
Learn enough Processing to set up examples.
(Re)learn JavaScript
Use Processing to ensure the visual results are the same.
Firefox, Internet Explorer (with Explorer Canvas), Opera, Safari, etc., on as wide a range of operating systems as possible, for testing

4.2 Version 0.2 (November 16)

4.2.1 What it to be done

Complete implementing multiple style set storage and conversion of existing code.
Implement pushStyle() and popStyle().

4.2.2 Gathering the pieces

Reference Processing code to ensure similar behaviour.
Set up a web page using canvas and Pjs for testing purposes.
Use Processing to ensure the visual results are the same.
Firefox, Internet Explorer (with Explorer Canvas), Opera, Safari, etc., on as wide a range of operating systems as possible, for testing

4.3 Version 0.3 (December 7)

4.3.1 What it to be done

Complete implementing pushStyle() and popStyle().
Firefox, Internet Explorer (with Explorer Canvas), Opera, Safari, etc., on as wide a range of operating systems as possible, for testing

4.3.2 Gathering the pieces

Reference Processing code to ensure similar behaviour.
Use Processing to ensure the visual results are the same.
Firefox, Internet Explorer (with Explorer Canvas), Opera, Safari, etc., on as wide a range of operating systems as possible, for testing

5 From here on in it's who you know

I briefly "spoke" to Al MacDonald (F1LT3R) on IRC (#processing.js on moznet). He had no objections and suggested pushStyle() and popStyle() might be placed after pushMatrix() and popMatrix() in the library, and that I should not hesitate to ask if I needed help.

6 There's no party like the...

I anticipate that others may assist in testing. I have learned from first hand experience that supposedly "identical" browsers on different operating systems may behave differently. (I recall sometime in the past that the "same" version of Firefox rendered a page differently on Windows than on Debian. In hindsight, it may have been something with the libraries Firefox depended on in Debian.) I do not have immediate access to the full range of combinations and it would be good to have a list of browser misbehaviour, even if making workarounds for some of them is impractical.

Then there is a matter of reviewing the code. There will no doubt be my unwanted idiosyncrasies and stealthy errors in the code, and these will be easier for others to pick out.

When my familiarization with the Pjs library improves I hope to have time to take cracks at some of the smaller functions that are still outstanding.

7 What could possibly go wrong?

There is no one element if the project, or combination thereof, that strikes me as requiring more than the usual caution. There are sufficient examples and documentation to make learning Processing and JavaScript, at worst, a trial-and-error proposition. Setting up new tools, like the Eclipse plugins, may involve some work, but if it goes badly they may be discarded without crippling the project; strictly speaking they are not essential and a normal text editor will do. Within the project I fear mainly the usual obscure bugs that resist solution, but these are not things that are generally "prevented" so much as "solved".

The greatest concern is when the project is considered in conjunction with other exterior projects. Already other courses are handing out assignments that are far more involved than my portion of Pjs. The question is not whether there is enough time to complete them all, but rather how many mornings I may resemble a shuffling zombie. I suppose it's a good thing I take public transit to school!

8 Once started, there is no going back

So this is it, open source software project number one! From this point shall I achieve fame, fortune, world domination...! Or I could just concentrate on getting through the semester and hoping my code passes muster. Yes, that sounds like a much better idea.

Let the good times roll.

Building the Minefield or: How I Put Some Under My Own Feet

2009-09-21T18:44:00.000-07:00

This week in DPS909 it was time to build the latest development version of Mozilla Firefox.

The first time I ever built something after checking out from a repository was a few years ago with the Icculus FreeSpace 2 Linux port. This was followed a bit later later with the FreeSpace 2 Source Code Project, and then somewhat more recently with the C-Dogs SDL port. For all of these configuring the system and building was a simple affair. Most of the configuration files, like ./configure and make and makefile, were already included with the checked out files and waiting to be run. The most trouble to be had was ensuring one had all the dependencies; this usually meant attempting to build multiple times, with each time following a fix to another missing dependency.

Building Mozilla Firefox was much the same, in which much of the trouble was fullfilling the dependencies. I started with the simple build instructions. Having casually browsed only that page the first attempt failed miserably because I had neglected to install autoconf. As we will soon see this would not be the only time my unfortunate habit of acting before browsing the literature with due diligence would haunt me.

Then the second attempt failed. The error message stated autoconf 2.13 was missing, but I assumed that any newer version would do, so I got autoconf 2.64 from the Debian GNU/Linux repository. This was a very poor assumption on my part. Finally getting to Linux build instructions it said that only 2.13 would work. Some Googling revealed the newer versions of autoconf were not backward compatible and Mozilla had stuck with 2.13 to avoid breaking and having to rewrite their scripts.

At that point I was getting that sinking feeling. I do not cherish the thought of cobbling together a package for Debian which may not coexist nicely with a newer version in the repository. After all, I might need the newer version sometime, and what if I had to get other older libraries to get the older autoconf to work? Fortunately, after some searching, I discovered Debian maintained autoconf 2.13 in the repository for the exact reason of backward compatibility.

So now I had autoconf 2.13 and (surprise, surprise!) the third build attempt also failed. The code in configure was malformed and gave this cryptic message like:

/home/mlam/src/mozilla-central/configure: line 4980: syntax error near unexpected token `}'   
/home/mlam/src/mozilla-central/configure: line 4980: `$as_echo "yes" >&6; }'

I admit I barely know the first thing about scripting and was not pleased with this since, I was fairly sure, this was not something I could fix by messing around with the script. Googling didn't turn up anything on this. Looking into the script the top comment said autoconf 2.64 was the generator. This got me worried: if the Mozilla script was using the wrong one something might be wrong with Debian's inner workings and this, again, was not something I relished poking and prodding. I read the Debian documentation for autoconf 2.13 on how which version to use was determined and as far as I could tell the Mozilla code was correctly setup to use 2.13.

So, going out on a limb and simply deleted the old configure and entered make -f client.mk yet again to regenerate. Fortunately that worked and got me past the problem.

Now, if I was in the habit of paying attention and reading the instructions, I could have avoided the previous problem. The error messages explicitly state to use

make -f client.mk build

when attempting to rebuild. It was only after solving the previous problem in that particularly ignorant fashion that I picked up on this.

The final problem was a run-of-the-mill missing dependency, in this case libidl.

Final time spend: over an hour puzzling through the build errors, and 20 minutes for the final successful build (naturally in my obstinance I insisted on using the system for other tasks at the same time.) So now Minefield is back on my system after a multi-year hiatus, although this time round I have no idea what I'm going to do with it.

However, all being said and done, building Firefox is ridiculously easy compared to the complexity of the program itself. No doubt a good number of people tore out hair getting the build infrastructure in place but the results are undoubtedly effective.

Now, if only Iceweasel 3.5.3 would enter the Debian Unstable repository...

Disembodied voices and talking heads!

2009-09-17T08:53:00.000-07:00

This past Wednesday I dialled into the Mozilla's weekly delivery meeting for Firefox and Gecko. Unfortunately, this required a long-distance call; my schedule constrained the meeting I could “attend”. On the up side, as an end-user I had some context to understand what was being discussed. Listening in on the developer meeting held Tuesdays might have been more arduous, if the wiki notes are anything to judge by.

It was of interest that the meeting's wiki page was used by the speakers. Each speaker edited in a summery of their points prior to, and sometimes almost right before, their turn to speak. One could tell who was absent simply by observing the empty sections on the page! Aside from leaving a record for the future, the wiki page content also helped me understand the point being discussed whenever it became difficult to hear what was being said; I suspect the meeting participants also appreciate the alternate channel the wiki provides.

Next I viewed a lecture by Mike Beltzner. He gave a brief overview of some of tools used for development and communications by Mozilla, such as Bugzilla and IRC. Beltzner also spoke about netiquette and how to go about communicating and asking for help, of respecting other people's time by being up front with what you need help with, doing the research before hand, and of course listening and implementing feedback. This was related to the brief section on “social capital” and the relationship between the influence of a person with the perceived value of the person by the community. I touched on this in a previous from a wiki sysop's perspective post where I related how a contributor's enthusiasm was no substitute for integration when seeking to become a valued member of a project.

Finally there was David Humphrey's lecture “Learning to be at the festival”. Humphrey emphasized the most important element of open source is the community rather than the code. The prominence of the code and product as the public face of the project makes it easy to forget that the participants, the community, are the project's fount and true source of influence. He also touched on ways to help new contributors submerge themselves into the project, such as existing contributors acting as mentors.

A milestone of sorts: first blog post ever.

2009-09-13T05:39:00.000-07:00

Greetings! My name is Matthew Lam, and I am a student in a software development program at Seneca College, Toronto, Canada. The main focus of the blog will be an open source project pursuant to achieving the objectives of an open source development course I am pursuing this semester (DPS909). The scope may broaden as time and whim permits.

As a lead-in to the course, I was required to read/listen/watch to a number of items concerning the nature and history of open source development. I have never participated in an open source software project before despite being familiar with the concepts and using the fruits of such endeavours myself. (Many years ago I was interested in the FreeSpace 2 Source Code Project but was fatally handicapped by a lack of knowledge of C/C++; now if I could just find the time...)

More recently most of my collaborative work has been as a sysop on a Wikia wiki. Imminently I face the reality that as part of DPS909 I will soon be experiencing a reversal of states. On the wiki part of the responsibility of determining the project's direction and environment is invested in me. However, soon I will be joining an established project where I will be the new contributor, not as a core driver.

In open source and wiki projects, treating users as co-developers is how the project gains person-power, but as I have learned it is practically a more complex matter. Acclimatizing new contributors is the challenge and the success of which lies mainly with the contributor. Contributors who really want to become valued members need little prompting to learn (by example, by asking for help, by heeding the advice of established members, etc.) how the project “works”; it is, after all, not enough to just contribute, one must contribute in the “right way”, according to the expectations prevalent within the project.

Having seen some new enthusiastic wiki contributors fail at the crucible, the main lesson is to be adaptable. The project may do things in ways not to one's liking but it is still contingent on oneself, at least at the beginning, to do things their way. To do otherwise makes it more difficult for one's contributions to be accepted.

Saying that, allow me to offer humble apologies in advance to all of the drivers/managers/etc. whom I may be a cause of distress for in the coming months. I know where you're coming from. :)