Wednesday, December 9, 2009

DPS909 Project – Processing.js – v0.3

And It's A Warp...
… for this phase. This is the last release for this semester's DPS909 project. For this bit strokeCap() and strokeJoin() were ported from Processing to Processing.js. The meat of what was done may be found in previous posts linked to above, so I won't occupy everybody again by repeating it here (after all, it's exam time!)
So without further ado, here are pertinent the pertinent links:

Links to Elsewhere
Links to the Past
And Away!
That's it for now. With a little luck I'll see you right back here in a months time for DPS911.

Wednesday, December 2, 2009

Processing.js – strokeJoin()

Copy and Pasting,,, Works!
Hot on the heels of strokeCap() strokeCap() is strokeJoin(). This blazingly fast turn around was accomplished by the time honoured method of shamelessly lifting previous work wholesale and tweaking.
Observe: here we have strokeCap():

p.strokeCap = function strokeCap( set ){
if ( set == p.ROUND ) {
curContext.lineCap = 'round';
} else if ( set == p.SQUARE ) {
curContext.lineCap = 'butt';
} else if ( set == p.PROJECT ) {
curContext.lineCap = 'square';
}
};


And here we have strokeJoin():

p.strokeJoin = function strokeJoin( set ){
if ( set == p.MITER ) {
curContext.lineJoin = 'miter';
} else if ( set == p.BEVEL ) {
curContext.lineJoin = 'bevel';
} else if ( set == p.ROUND ) {
curContext.lineJoin = 'round';
}
};


As we can see strokeCap() gained a very close sibling in strokeJoin()!

Newborn Boo-boos
This is not to say the birth of the younger didn't affect the elder. Originally the constants for strokeCap() and strokeJoin() were defined separately, like so:

// Stroke cap constants.
p.ROUND = 0;
p.SQUARE = 1;
p.PROJECT = 2;

// Stroke join constants.
p.MITER = 0;
p.BEVEL = 1;
p.ROUND = 2;


p.ROUND was redefined for strokeJoin() and ended up conflicting with strokeCap()'s p.PROJECT. This caused... incorrect behaviour in an otherwise well behaved strokeCap().
The problem was solved simply be consolidating the constants into:

// Stroke cap and join constants.
p.ROUND = 0; // Used by both cap and join.
p.SQUARE = 1; // Used by cap.
p.PROJECT = 2; // Used by cap.
p.MITER = 3; // Used by join.
p.BEVEL = 4; // Used by join.


Pretty Pixels!
To end off here is the code in action compared to the results of the same code in the Processing Development Environment (PDE).



On a side note I've noticed the Processing website's examples are imprecise; their code and images do not match up. The images show the result of strokeCap(SQUARE) or strokeCap(PROJECT) being called, but these calls are missing in the example code.

Sunday, November 29, 2009

Processing.js – strokeCap()

strokeCap()
Another week or two, and a bit of work toward the 0.3 release of my DPS909 project. This time it was strokeCap()'s turn to be ported to Processing.js (Pjs).
So here is the basic Processing example working in Pjs:


And here is the example running in the Processing Development Environment (PDE):

Squishy Innards
This is all done by setting canvas' lineCap attribute. lineCap has three possible values which correspond directly to those used by Processing so strokeCap()'s code looks just like this:
p.strokeCap = function strokeCap( set ){
if ( set == p.ROUND ) {
curContext.lineCap = 'round';
} else if ( set == p.SQUARE ) {
curContext.lineCap = 'butt';
} else if ( set == p.PROJECT ) {
curContext.lineCap = 'square';
}
};

lineCap is also handled by canvas' state saving functions, which means Pjs' pushStyle() and popStyle() can already handle saving and restoring the setting as well.

A Different Starting Point
The main difference between canvas and Processing/Pjs is the default setting for lineCap. Canvas' default is “butt", which is “SQUARE" in Processing. Processing needs to default to “ROUND", which is “round" in canvas.

There was a bit of trouble getting the default working in Pjs. At first adding curContext.lineCap = 'round'; to function init didn't work. Al MacDonald (F1LT3R) was very quickly able to find where the problem was and fix it: it turns out function size recreated the canvas context every time it was called, which in turn reset all the attributes. Thanks, Al!

So now the default setting works very nicely in Pjs:


Compare to the same thing in PDE:

Useful Links

Thursday, November 26, 2009

Firefox Addon and XPCOM

So last week in DPS909 we were supposed to feel our way through a lab giving some idea how Mozilla's XPCOM works. XPCOM is used to ensure code portability. I do not pretend to understand how it all works, it's rather complicated and terrifying for a newbie, but at the very least I can follow instructions.

The big first step is to create an XPCOM interface using a XPIDL file, which includes function prototypes using familiar C/C++ data types (at least in the walkthrough, XPCOM may be used with other languages to). A header file is automatically generated which, amongst other things, transforms the data types into Mozilla's data types and creates a stub class using the header. In other words, a lot of the basic set up work is done for the code cruncher (winner!)

Most of the walkthrough was pretty straight forward. There were a few minor things that needed to be changed or were different. In install.rdf, I changed em:minVersion and em:maxVersion to 3.5.5 and 3.7a1pre respectively (that allowed the extension to work in the trunk build.) Also, IFirstXpcom,h was created in $(objdir)/dist/include/ rather than $(objdir)/dist/include/firstxpcom/.

But everything worked out and so I can now claim to have created (after near slavishly following somebody else's instructions) an add-on for Firefox.


Friday, November 20, 2009

Processing.js – Something Lost/Gained in the Translation

The nice thing about porting is the end result is already known. Comparing the effects of the original with the effects of the transcription is a good indicator of how close you are and whether or not you're on the right track.

Simple, yes?

Sure, at least in a perfect world where I could have my kitchen faucet yield hot chocolate and have my computer magically upgrade itself to the latest hardware and software. Alas, a perfect, honest-to-the-source, zero deviation transcription may not always be possible; such is what I have discovered while tinkering with pushStyle() popStyle() and boolean().

Below you will find I have not embedded the Pjs examples, instead I shall provide a link to another web page with the example on it. This is because embedding requires minification, and I'd rather not go through the frustration of minifying some of the larger examples. I'll provide screenshots of how the examples look using the Processing Development Environment (PDE) for comparison.

Differences in pushStyle() and popStyle()
The basic Processing reference example looks like this

and can now be reproduced in Pjs (Pjs demo here.)

But take a Pjs demo like this one. See the extra pops at the bottom? You can't do that in Processing. The PDE won't allow the Pjs example code to run; it'll see the extra pops, throw an exception, and refuse to run properly until the extra pops are removed. The closest approximation to the Pjs example Processing will allow looks something like this:

Pjs doesn't really concern itself with catching logical errors in the code. At best it just concerns itself with syntactic errors, and if there are any it simply doesn't paint anything onto the canvas. Saying that, the Pjs version of popStyle() has to be a bit more lax than its Processing counterpart and work with, rather than crash on, extra pop calls.

In this matter the solution is simple. Upon figuring out the pop call is extraneous, Pjs effectively ignores it and carries on. It must be one of the few times burying one's head in the sand actually makes the problem go away!
Differences in boolean()
Moving onto boolean(). Let's start out with showing a basic example based on the one found on the Processing website. It looks like this:


and it too may be reproduced in Pjs (Pjs demo here.) However, boolean() is afar more complicated beast than the basic example would imply. In this case it's a matter of how Java/Processing is very strict on the number and nature of arguments passed to a method, while JavaScript/Pjs is not.

Take for example the following this Pjs demo. Note how we're passing in doubles, booleans, ints, nulls, undefines, etc., and arrays with mysteriously empty elements. If you were to try the code in Processing it wouldn't work. For one, Processing's boolean() refuses to handle boolean variables, or double variables, or nulls, and other fancy things like that. In fact, the closest Processing example to the Pjs demo is rather more sparse and looks like this:


(Un)fortunately, Pjs' boolean() can't take the easy way out and throw away arguments it doesn't like, coughing up an exception, and making the whole show grind to a halt. Being JavaScript it's syntactically correct to pass a cornucopia of dynamically typed arguments into a function. Pjs' boolean() has to handle whatever comes its way gracefully.

As you can tell from the Pjs boolean() demo, I thought this was best done by having the function return "meaningful" results from arguments not normally handled by Processing's boolean(). Numeric arguments are handled like integers: zero values return false, non-zero values return true. Boolean arguments are just returned as themselves. Nulls and undefines return false.

Heresy!
I like to think I've cobbled together an acceptable compromise between how Pjs and Processing behave. I guess I'll find out soon whether this is acceptable or not when I try to push the code into the library.

Thursday, November 19, 2009

DPS909 Project – Processing.js – v0.2

Returning to Ground Zero
First, a self-plagiarized recap. This semester at Seneca College I am taking a course revolving around open source development. As part of the course the students (yours truly being no exception) were nudged into getting involved in some Mozilla related projects, produce code for the real world, and see fist hand how the whole open source thing works.

I chose to work on Processing.js (Pjs), a JavaScript port of the Java-based Processing. More specifically, I volunteered to work on porting over the functions pushStyle(), popStyle(), and boolean().

On 19 October 2009, I threw release 0.1 out into the cruel world to fend for itself. Today I present release 0.2 to the world. Applause!

Not new, but shinier!
The watchwords for release 0.2 were: fix it. Release 0.1 included reasonably functional code for pushStyle() and popStyle(), and totally untested and untried code for boolean().

Bringing boolean() up-to-spec was the goal of this release and this was done by throwing together some test cases for boolean(). Bugs were indeed caught, squashed, and flushed down the toilet.

Although boolean() got most of the attention, the elder siblings did not totally escape notice. pushStyle() and popStyle() received their own test cases too. One issue was whether the pop should spit out alert popups if there were not enough pushes. I decided not to use any popups and just have a warning be sent to the console, where most users probably won't notice. pushStyle() and popStyle() were further improved to better handle the library's own variables; before they only managed canvas' variables with any proficiency.

Better to get it when it's pleasantly warm rather than skin-charring hot

What now?
Pursuant to getting my code merged into the Pjs library there are a few instructions I have to follow. So I'll make another post comparing the results of my Pjs code with those of the Java-based Processing code. As I have discovered it is not totally possible to faithfully port Processing to JavaScript. For example, Processing will throw exceptions and refuse to do anything if you try to pass an argument to a function and the function is not designed to accept, say, the variable type of the argument. But in Pjs you have to handle these weird arguments and preferably return some meaningful result. This comes up a few times in boolean(). But more on that soon.

For release 0.3, it looks like I'll have to choose another function to port over, or at least get started on even if I don't finish by the end of the semester. strokeCap() looks appropriately involving.

Saturday, November 7, 2009

Learning a Bit of JavaScript

This week in DPS909 we talked a bit about JavaScript. This was particularly relevant to my project this semester, implementing a few functions in the Processing.js JavaScript library. As part of this week's journey of the mind, we were directed to watch a lecture by Douglas Crockford called "JavaScript: The Good Parts". Leaping over to Wikipedia for some familiarization, Crockford is the senior JavaScript Architect at Yahoo!, and wrote the specification for JavaScript Object Notation. The lecture in the video is based off his book of the same name.

Right off the bat, I'm one of those people who, as Crockford' puts it, started tinkering with JavaScript without learning it. So given my novice credentials coupled with a pressing need to "learn more" I'm pleased that the lecture had a few "do and don't" pointers that someone like me could take away.

The first big thing I learned was use ===, avoid ==. I was first exposed to === by David Humphrey, who gave me a code snippet demonstrating how to detect if a JavaScript variable was an array. A bit of research revealed === compares not only the value of two variables but their types as well, a very useful feature in a language with dynamic typing. I was, not, however, inclined to use === near exclusively.

After watching Crockford's lecture made me realize using mainly == would eventually cause grief.. Sooner or later I would make compare two values, forget the variable types, or forget the distinction between null and undefined, and end up with an annoying, very small, difficult to track down, bug. Using === would be an easy way to save a bit of heartache in the future, which is always a good thing.

Learning the distinction between null and undefined was an ancillary to the first major thing I picked up. The former is a value for an object variable, the latter can be had if trying to retrieve a variable an object doesn't have. Note to self: further research required.

The second, and most surprising, thing I learned was coding style matters. It's easy to mistake JavaScript for a free-wheeling language: dynamic typing, you don't have to use semi-colons half the time! But apparently something as "simple" as bracket placement when defining blocks can really mess things up. Crockford gave the following example:

return
{
ok: false
};

This seems innocent enough, but apparently behind-the-scenes shenanigans means it actually is interpreted like so:

return;
{
ok: false;
}

In other words, everything after return goes nowhere and does nothing. For this to work as intended the opening bracket must be placed to the right of return like so:

return {
ok: false
};

While my coding style generally resembles the "correct" example, this will be a useful tidbit when deciphering the code of other programmers.

Fortunately, it seems Crockford has developed JavaScript code quality checker called JSLint to help developers catch some of their more egregious code. Methinks I shall be availing myself of it in the near future.

Saturday, October 31, 2009

Standing On Their Shoulders: A Short Processing.js Tale

So this semester's reading week has come and gone and once again, as is the familiar post-mortem assessment, I have discovered the ratio of “useful" to “superfluous" work done is heavily biased toward the latter.

On the Processing.js (Pjs) front we have more baby steps!

Don't mind if I do...

For pushStyle() and popStyle() I've come full circle. First I fiddled around with storing and retrieving variables in the library, then I moved onto storing and retrieving canvas variables, and have since returned to working on the Pjs library variables.

In my previous post, I discovered some code by Andrew Peace. As it turns out, not only did Peace's work handle the Pjs variables, he also did it in a more JavaScript savvy way then I would have done.

The example that sticks out in my mind was his handling of arrays. While I was busy incrementing and decrementing a counter to figure out how many states I had and which element to store and retrieve from, Peace simply used the push() and pop() functions that grace all JavaScript arrays.

I have incorporated Peace's code with only cosmetic changes, and JavaScript's push() and pop() functions for arrays has since entered my corpus of All Knowledge. My hat off to Mr. Peace!

But does it work?

In my previous post I also intimated that some work was going on with boolean(). There was some code but no testing. In the last little bit I've started cobbling together a few test cases, and even thought still incomplete they've already helped flush out a few bugs.

The nice test page layout was provided by classmate Daniel Hodgin. Thanks!

Monday, October 19, 2009

DPS909 Project - Processing.js – v0.1

Last month on [insert name of media production here]...

First, a brief recap. This semester at Seneca College I am taking a course revolving around open source development. As part of the course the students (yours truly being no exception) were nudged into getting involved in some Mozilla related projects, produce code for the real world, and see fist hand how the whole open source thing works.

I chose to work on Processing.js (Pjs), a JavaScript port of the Java-based Processing. More specifically, I volunteered to work on porting over the functions pushStyle(), popStyle(), and boolean().

I've got to release something...!

Today I get to make my first (course mandated) release! And this is what it does (trotting out this little “demo" for the third time):




What's that? That, fine feathered friends, is pushStyle() and popStyle() working just a little bit. Nothing particularly amazing, it's just the Processing example copied verbatim.

A start on the boolean() code is also in the release, but it's not tested at all.

For those of you who want to see for yourself may:

Toward 0.2

The march to enlightenment continues, and there is still much to do. The implementation of pushStyle() and popStyle() remains incomplete as demonstrated by this discussion on the Pjs Google Group. So far I have only saved the state of canvas variables, and not those in the library itself.

Then I need to see about putting through a more expansive test regimen. And there's already a bug: using Andrew Peace's test case, the “more pops than pushes" error alert just keep popping up. I like to think the intended behaviour is for a finite number of warnings.

On the other hand, boolean() seems much more straight forward to test.

A not-so-secret society

v0.1 releases just in time for this years Free Software and Open Source Symposium hosted by Seneca College. This is particularly important because Al MacDonald, Pjs project leader, is attending FSOSS and will no doubt want to have a few words with those DPS909 students working on Pjs.

David Humphrey suggested us students might want to look into setting up a Pjs bug tracker, discuss ways to compile our seperate test suites into one big suite, and perhaps set up a website to showcase the newly ported capabilities. I figure even if I somehow manage to get my project done way ahead of schedule there will not be a lack of things to do.

Saturday, October 17, 2009

Fixing a Bug In A Harmless Test Environment

This week's DPS909 lab brought together everything covered thus far in the course: build the software, file a bug, find the offending code, make a patch. If this were any other course this lab might qualify as a "quest": not a quiz, not a test, but somewhere delightfully in between!

Step one was compiling a debug build of Thunderbird/Shredder. I checked out straight from the repo rather than use the code provided by the lab instructions. The process was similar to building Firefox/Minefield and went off literally without a hitch. A very palpable hit for me!

Nest was to confirm the bug's existence. The bug we were interested in deals with Thunderbird converting a string that looks like an email address into a hyperlink; the problem is the string does not represent a valid email address and so should not be converted. We can see the fubared behaviour below:


So having confirmed the existence of the bug without difficulty, bug #7243 was filed in the landfill. I like to think I included the bare-bones information in the description answering the questions of: "what is wrong?", "how may it be reproduced?", and "what version of the software are you using?".

Now came the matter of tracking down that needing futzing with. This took a while. I spent much time plopping in search terms into the Mozilla Cross-Reference without success. I looked into "parse message", "save draft", and the like to try and get into where the message was stored and perhaps displayed to the screen.

In desperation I went back to the program and, mousing over the offending email link, saw it was a mailto link. So I through in mailto as the search term and... there was light. (Just goes to show simplicity is golden.) Conveniently the very first result from that search took me exactly where I needed to go.

if (inString.FindChar('.', pos) != kNotFound)

The conditional only checked if a "." was found after the @ sign. What it also needed to do was check if there were any ".." after the @ too, which would mean it was not a valid address. Some way needed to be found to find whether a ".." was in the string. Unlike what was currently in mozTXTToHTMLConv.cpp I needed a way to check the presence of a string in the string, and not just a char in the string. According to the Mozilla internal string guide the correct way to find if a substring existed was to useFindInReadable in nsScannerString.cpp, However converting the code to use the newer approved way was probably a bit beyond what I wanted to delve into for this lab.

So I just stuck with the old way. Searching through the documentation some more I found nsDependentString had just the function I needed. The code mozTXTToHTMLConv changed to:

if (inString.FindChar('.', pos) != kNotFound && inString.Find("..", pos) == kNotFound)

A recompile later the situation was much improved:


I created the patch from ./comm-central/mozilla and ran it through JST Review. A few "longer than 80 characters" and extra whitespace/tabs warnings later, it was ready to be attached to the bug report.

Ehren graciously took some time to review the patch and found I had indented a line a bit too far. It was up to a patched patch to pass muster instead.

The tool-tips for the review options for attachments threw me off a bit. Super-reviews are only needed for world-shaking changes, but the super-review tool-tip says something somewhat different. I didn't request a super-review for the revised patch.

The patch and review comments have been copied to my Seneca College wiki page.

Monday, October 12, 2009

boolean() for Processing.js

And so we return for another (short) post in the irregular series on “What is Matthew doing with Processing.js?" Ah, the joys of lagging a bit behind from where where I should be. But onwards to putting a little bit more on my plate!

Seeking truth From variables

There is a spectre haunting the incompleteness of Processing.js (Pjs) (well, one of many spectres) - the spectre of boolean(). It is a spectre I hope to, at least, partially exorcise by December.

This little function takes a variable, may it be integer, String, object, or an array thereof, and spits out “true" of “false" (or even an array of such). A little peeking into the original Java code shows that the function is actually spread over five overloaded Java methods in class PApplet.

Of course, it will all have to be condensed into a single function in Pjs. Since it doesn't seem JavaScript requires the explicit declaration of data types, I'll have to search around to see how one might go about distinguishing between them. However I expect there will be plenty of examples to sink my teeth into. (I can't be the first person to ask: “How do I tell if a variable is an array?" after all!)

The project page has been given a minor revision to reflect the new bit on the plate. I really must get around to jotting down some documentation on my latest work with pushStyle() and popStyle(). See? Lagging behind.

Speaking of which...

In my previous post I displayed a basic working example for pushStyle() and popStyle(). Alas, I hosted the JavaScript library on a temperamental web host so it can't be seen half the time.

I redisplay the example here using (hopefully) a more reliable web host.



But where has the time gone? Release version 0.1 is just around the corner, and there is still things to ponder and code to crunch! I guess it would not be a student's life if one did not feel kinship to the tardy White Rabbit.

Saturday, October 3, 2009

According to the Plan

… this shouldn't be happening.

In a previous post outlining my DPS909Processing.js (Pjs) project plan, I sketched out the implementation of pushStyle() and popStyle() over a period of over two months. As it turns out this may have been overly pessimistic.

Climbing the Stairs

In the beginning, the scheme called for storing and retrieving states from a nice 2D array. I opted for an object to store each state snapshot.I peeked into the Java Processing code (PStyle.java) to see what sort of variables I might be looking for. The variables listed under the comment // “Private" variables used to maintain state seemed like a good place to start.

Throwing the code into a test page, let's just say those were not, or not all, the variables I was looking for.

Finding the Elevator

Closer examination revealed a mysterious object stored as curContext, and it seemed the object's squishy innards contained what I was looking for. Since those innards didn't seem to be defined in the Pjs library I turned to Google, and that turned up the canvas element documentation.

As it turns out canvas comes fully equipped with functions to save and retrieve state snapshots. So using those functions instead of the array, and state-saving objects, we get something like:



Assuming one's browser comes with the requisite bells and whistles, it should look suspiciously similar to the Processing example.

Expanding the Catch

In terms of getting pushStyle() and popStyle() to work, there are a few things I still need to look into. In my current revision I am not handling any of the library's own variables at all. The basic example works without them but, but some may be relevant state variables and I have to figure out which ones (if any) are so.

Another concern is browser compatibility. Canvas implementation seems to be all across the board right now. If Pjs has to work now, without reservations, then I'll have to go back to the state-saving object and array scheme, while saving all of the canvas variables manually. On the other hand, if the onus is not on Pjs to work on browser's with incomplete canvas implementations, then that's one less thing to worry about.

But the bottom line is that the project as I envisioned it will most likely be done long before early December. This is not acceptable for DPS909 and therefore I must find additional bits of Pjs to hack away at. I'll be figuring that out soonish and changing altering my wiki project page to reflect that.

Monday, September 28, 2009

My Kingdom for a Patch?

This blog post is brought to us by the letter P and the number 6, and today's topic is “patching". “Patching". (Alas the written word doesn't do justice to the beginning of each Possum Lodge Word Game.)
Patches are funny “little" files that contain the list of changes or differences between two snapshots of a code base. This is good for keeping track of changes to the code, facilitating reversions, and making it easier to spread the joy of testing and breaking systems to self-appointed guinea pigs.

For this DPS909 lab, the first thing to do was hunt for a patch for my Minefield build. I headed over to the Firefox Builds forum at mozillaZine. From Peter(6)'s seemingly endless series on the rise and fall of the nightly trunk builds, I picked out recently resolved bug 424517.

The gist of 424517 was the “use small icons" option was affecting the additional icons added to the Bookmarks Toolbar, when such icons should be start small and remain small no matter what.
I verified that my Minefield and Iceweasel (just because I could) were exhibiting the unfortunate symptoms of the issue. The “refresh" icon in the Bookmarks Toolbar did indeed resize as the “use small icons" option was toggled back and forth.

The patch added a mere two lines, but the important thing was the results could be easily verified. So I applied it:

mlam@mlamprimary:~/src/mozilla-central$ patch -p1 < lockiconsize.diff
patching file browser/base/content/browser.xul


After some rebuilding I prodded open my improved version of Minefield and, lo and behold, it was cured! The “use small icons" had no effect on the Bookmarks Toolbar, just as the Givers Of Data deemed it should be (and there was much rejoicing.) But just to be perverse:

mlam@mlamprimary:~/src/mozilla-central$ patch -R -p1 < lockiconsize.diff
patching file browser/base/content/browser.xul


And after another bit of rebuilding the issue returned as mysteriously as it had vanished. Very sad.

Sunday, September 27, 2009

Processing.js – A Plan to Push and Pop Styles

1 What this is all about

Processing is a Java-based programming language used to display rich graphical content, like animations, on web pages. The content was embedded on web pages using Java applets. Since the time of the original implementation improvements in web technology, such as the capability of web browsers and JavaScript execution speed, has opened the possibility of accomplishing Processing's objective without Java.

Processing.js (Pjs) is a JavaScript port of Processing currently in development. Users need only a sufficiently capable web browser to view the content; there is no need for Java or Flash, their plugins, and the overhead and possible software troubles arising from using those plugins. Matters are also simplified for developers who only need Pjs's JavaScript library.

A convenient collection of links to mailing lists and code repository may be found here. A specific list of "things needing code monkey labour" may be found on MozillaWiki

2 Choosing the road frequently travelled

For DPS909 I and my fellow code monkeys-in-training were directed to jump into a Mozilla-related project. As the old (and perhaps overused) saying goes: I hear and I forget, I see and I remember, I do and I understand. Appropriately for a course about open source development, we were going to do it, and, hopefully, emerge from the experience with greater understanding.

One available set of projects fell under Pjs. Within that project I chose to work on porting pushStyle() and popStyle() from Processing to Pjs.

Before going on I should disclose that I like Java. It's the first object-oriented language I picked up. At the very least, reading the code to be ported would be easier. However, Java was not the major reason for my project choice; that language plays a relatively diminutive part in the overall task. The major part is reading and writing JavaScript which. I have very little experience with JavaScript and working on Pjs presented an opportunity to rectify this for fun and profit! Oh, and there would be ancillary social benefits such as being useful in the process.

Aren't win-win situations wonderful?

I chose tasks from the "miscellaneous feature" list rather than the "3D features" list because the latter constitutes a "here be dragons" area for me. When the course concludes and my work is evaluated, I would rather submit better material and feel silly for choosing something that did not wander far from my starting experience, than submit sub-par material and experience the self-recrimination of having barrelled toward the "bridge too far".

Discretion is the better part of valour.

3 The information must flow

So that a record of progress, U-turns, and backtracking, may be preserved for future generations to point and snicker at, I will be maintaining a wiki page. There is also a branch of the development tree. Hopefully there will be weekly posts on this blog concerning the project work.

4 If everything is going according to plan, it's a trap

As per the requirements for DPS909 the aim is for three "point releases" within the next two and a half months. This means version 0.3 will be available in early December. Per the expectations of the course it is not intended to be the final product although one hopes it will be of sufficient quality for somebody to pick up where I leave off, providing I do not take DPS911 next semester.

The following is a brief overview of the objectives and predicted issues for the point releases.

4.1 Version 0.1 (October 19)

4.1.1 What it to be done
  • Add a 2D array of style variables with each "row" representing a style set.
  • Add a variable tracking the style set currently in use. This will be technically superfluous during the interim since only a single style set will be used.
  • See if we need functions to save the current style variable values or load a set into the variables. Might be overkill.
4.1.2 Gathering the pieces

4.2 Version 0.2 (November 16)

4.2.1 What it to be done
  • Complete implementing multiple style set storage and conversion of existing code.
  • Implement pushStyle() and popStyle().
4.2.2 Gathering the pieces
  • Reference Processing code to ensure similar behaviour.
  • Set up a web page using canvas and Pjs for testing purposes.
  • Use Processing to ensure the visual results are the same.
  • Firefox, Internet Explorer (with Explorer Canvas), Opera, Safari, etc., on as wide a range of operating systems as possible, for testing

4.3 Version 0.3 (December 7)

4.3.1 What it to be done
  • Complete implementing pushStyle() and popStyle().
  • Firefox, Internet Explorer (with Explorer Canvas), Opera, Safari, etc., on as wide a range of operating systems as possible, for testing
4.3.2 Gathering the pieces
  • Reference Processing code to ensure similar behaviour.
  • Use Processing to ensure the visual results are the same.
  • Firefox, Internet Explorer (with Explorer Canvas), Opera, Safari, etc., on as wide a range of operating systems as possible, for testing

5 From here on in it's who you know

I briefly "spoke" to Al MacDonald (F1LT3R) on IRC (#processing.js on moznet). He had no objections and suggested pushStyle() and popStyle() might be placed after pushMatrix() and popMatrix() in the library, and that I should not hesitate to ask if I needed help.

6 There's no party like the...

I anticipate that others may assist in testing. I have learned from first hand experience that supposedly "identical" browsers on different operating systems may behave differently. (I recall sometime in the past that the "same" version of Firefox rendered a page differently on Windows than on Debian. In hindsight, it may have been something with the libraries Firefox depended on in Debian.) I do not have immediate access to the full range of combinations and it would be good to have a list of browser misbehaviour, even if making workarounds for some of them is impractical.

Then there is a matter of reviewing the code. There will no doubt be my unwanted idiosyncrasies and stealthy errors in the code, and these will be easier for others to pick out.

When my familiarization with the Pjs library improves I hope to have time to take cracks at some of the smaller functions that are still outstanding.

7 What could possibly go wrong?

There is no one element if the project, or combination thereof, that strikes me as requiring more than the usual caution. There are sufficient examples and documentation to make learning Processing and JavaScript, at worst, a trial-and-error proposition. Setting up new tools, like the Eclipse plugins, may involve some work, but if it goes badly they may be discarded without crippling the project; strictly speaking they are not essential and a normal text editor will do. Within the project I fear mainly the usual obscure bugs that resist solution, but these are not things that are generally "prevented" so much as "solved".

The greatest concern is when the project is considered in conjunction with other exterior projects. Already other courses are handing out assignments that are far more involved than my portion of Pjs. The question is not whether there is enough time to complete them all, but rather how many mornings I may resemble a shuffling zombie. I suppose it's a good thing I take public transit to school!

8 Once started, there is no going back

So this is it, open source software project number one! From this point shall I achieve fame, fortune, world domination...! Or I could just concentrate on getting through the semester and hoping my code passes muster. Yes, that sounds like a much better idea.

Let the good times roll.

Monday, September 21, 2009

Building the Minefield or: How I Put Some Under My Own Feet

This week in DPS909 it was time to build the latest development version of Mozilla Firefox.

The first time I ever built something after checking out from a repository was a few years ago with the Icculus FreeSpace 2 Linux port. This was followed a bit later later with the FreeSpace 2 Source Code Project, and then somewhat more recently with the C-Dogs SDL port. For all of these configuring the system and building was a simple affair. Most of the configuration files, like ./configure and make and makefile, were already included with the checked out files and waiting to be run. The most trouble to be had was ensuring one had all the dependencies; this usually meant attempting to build multiple times, with each time following a fix to another missing dependency.

Building Mozilla Firefox was much the same, in which much of the trouble was fullfilling the dependencies. I started with the simple build instructions. Having casually browsed only that page the first attempt failed miserably because I had neglected to install autoconf. As we will soon see this would not be the only time my unfortunate habit of acting before browsing the literature with due diligence would haunt me.

Then the second attempt failed. The error message stated autoconf 2.13 was missing, but I assumed that any newer version would do, so I got autoconf 2.64 from the Debian GNU/Linux repository. This was a very poor assumption on my part. Finally getting to Linux build instructions it said that only 2.13 would work. Some Googling revealed the newer versions of autoconf were not backward compatible and Mozilla had stuck with 2.13 to avoid breaking and having to rewrite their scripts.

At that point I was getting that sinking feeling. I do not cherish the thought of cobbling together a package for Debian which may not coexist nicely with a newer version in the repository. After all, I might need the newer version sometime, and what if I had to get other older libraries to get the older autoconf to work? Fortunately, after some searching, I discovered Debian maintained autoconf 2.13 in the repository for the exact reason of backward compatibility.

So now I had autoconf 2.13 and (surprise, surprise!) the third build attempt also failed. The code in configure was malformed and gave this cryptic message like:

/home/mlam/src/mozilla-central/configure: line 4980: syntax error near unexpected token `}'
/home/mlam/src/mozilla-central/configure: line 4980: `$as_echo "yes" >&6; }'


I admit I barely know the first thing about scripting and was not pleased with this since, I was fairly sure, this was not something I could fix by messing around with the script. Googling didn't turn up anything on this. Looking into the script the top comment said autoconf 2.64 was the generator. This got me worried: if the Mozilla script was using the wrong one something might be wrong with Debian's inner workings and this, again, was not something I relished poking and prodding. I read the Debian documentation for autoconf 2.13 on how which version to use was determined and as far as I could tell the Mozilla code was correctly setup to use 2.13.

So, going out on a limb and simply deleted the old configure and entered make -f client.mk yet again to regenerate. Fortunately that worked and got me past the problem.

Now, if I was in the habit of paying attention and reading the instructions, I could have avoided the previous problem. The error messages explicitly state to use

make -f client.mk build

when attempting to rebuild. It was only after solving the previous problem in that particularly ignorant fashion that I picked up on this.

The final problem was a run-of-the-mill missing dependency, in this case libidl.

Final time spend: over an hour puzzling through the build errors, and 20 minutes for the final successful build (naturally in my obstinance I insisted on using the system for other tasks at the same time.) So now Minefield is back on my system after a multi-year hiatus, although this time round I have no idea what I'm going to do with it.

However, all being said and done, building Firefox is ridiculously easy compared to the complexity of the program itself. No doubt a good number of people tore out hair getting the build infrastructure in place but the results are undoubtedly effective.

Now, if only Iceweasel 3.5.3 would enter the Debian Unstable repository...

Thursday, September 17, 2009

Disembodied voices and talking heads!

This past Wednesday I dialled into the Mozilla's weekly delivery meeting for Firefox and Gecko. Unfortunately, this required a long-distance call; my schedule constrained the meeting I could “attend”. On the up side, as an end-user I had some context to understand what was being discussed. Listening in on the developer meeting held Tuesdays might have been more arduous, if the wiki notes are anything to judge by.

It was of interest that the meeting's wiki page was used by the speakers. Each speaker edited in a summery of their points prior to, and sometimes almost right before, their turn to speak. One could tell who was absent simply by observing the empty sections on the page! Aside from leaving a record for the future, the wiki page content also helped me understand the point being discussed whenever it became difficult to hear what was being said; I suspect the meeting participants also appreciate the alternate channel the wiki provides.

Next I viewed a lecture by Mike Beltzner. He gave a brief overview of some of tools used for development and communications by Mozilla, such as Bugzilla and IRC. Beltzner also spoke about netiquette and how to go about communicating and asking for help, of respecting other people's time by being up front with what you need help with, doing the research before hand, and of course listening and implementing feedback. This was related to the brief section on “social capital” and the relationship between the influence of a person with the perceived value of the person by the community. I touched on this in a previous from a wiki sysop's perspective post where I related how a contributor's enthusiasm was no substitute for integration when seeking to become a valued member of a project.

Finally there was David Humphrey's lecture “Learning to be at the festival”. Humphrey emphasized the most important element of open source is the community rather than the code. The prominence of the code and product as the public face of the project makes it easy to forget that the participants, the community, are the project's fount and true source of influence. He also touched on ways to help new contributors submerge themselves into the project, such as existing contributors acting as mentors.

Sunday, September 13, 2009

A milestone of sorts: first blog post ever.

Greetings! My name is Matthew Lam, and I am a student in a software development program at Seneca College, Toronto, Canada. The main focus of the blog will be an open source project pursuant to achieving the objectives of an open source development course I am pursuing this semester (DPS909). The scope may broaden as time and whim permits.

As a lead-in to the course, I was required to read/listen/watch to a number of items concerning the nature and history of open source development. I have never participated in an open source software project before despite being familiar with the concepts and using the fruits of such endeavours myself. (Many years ago I was interested in the FreeSpace 2 Source Code Project but was fatally handicapped by a lack of knowledge of C/C++; now if I could just find the time...)

More recently most of my collaborative work has been as a sysop on a Wikia wiki. Imminently I face the reality that as part of DPS909 I will soon be experiencing a reversal of states. On the wiki part of the responsibility of determining the project's direction and environment is invested in me. However, soon I will be joining an established project where I will be the new contributor, not as a core driver.

In open source and wiki projects, treating users as co-developers is how the project gains person-power, but as I have learned it is practically a more complex matter. Acclimatizing new contributors is the challenge and the success of which lies mainly with the contributor. Contributors who really want to become valued members need little prompting to learn (by example, by asking for help, by heeding the advice of established members, etc.) how the project “works”; it is, after all, not enough to just contribute, one must contribute in the “right way”, according to the expectations prevalent within the project.

Having seen some new enthusiastic wiki contributors fail at the crucible, the main lesson is to be adaptable. The project may do things in ways not to one's liking but it is still contingent on oneself, at least at the beginning, to do things their way. To do otherwise makes it more difficult for one's contributions to be accepted.

Saying that, allow me to offer humble apologies in advance to all of the drivers/managers/etc. whom I may be a cause of distress for in the coming months. I know where you're coming from. :)