Gears as a prototype platform

So the blog has been silent for a week.  I was hoping to not let that happen so early on, but I only have so much free time, and I have to divide it between getting things done and talking about getting things done. So here’s the scoop on what I’ve been working on:

I’ve been researching a lot of different areas to at least get some goals and rudimentary plans together.  In general, I’ve been trying to find all of the areas that can be used in some way to prototype the behavior I’m looking for.  The main code bases that I’m looking into are Gears and Webkit.  I’ve also been spending a lot of time with HTML5 and SVG.  At this point, I’m just a little overwhelmed.  The scope is obviously huge and I’m not sure where to start, but I think I’ve got some good ideas.

The First Step

As only one guy for now, I just want to get the minimum effort going that is required to get the project off of the ground.  I think the easiest way to start is to begin with javascript language improvements.  The easiest way to begin that is to build a translator.  So here’s the plan: I’m mostly a Java/JavaScript programmer and the easiest tool for me to use is ANTLR. It’s extremely fast to build grammars with and I think translating to existing JavaScript a la GWT will be a very fast approach.  Aspects of the new javascript such as actor style concurrency can be simulated using the Gears worker pool.

At first, I will just translate on the server, but the next step can be to create a new Gears module that will perform the translation on the client.  When a good working model is established the next steps can be decided from there.

Future steps

At some point in the future, a visual component of the plugin will be required.  Gears has nothing of the kind right now.  This will be a much bigger challenge than just having another language runtime.

While I don’t want to just settle for what is available, I do think that building off of existing technologies is much more realistic than starting completely from scratch.  I think that SVG could be a very good building block for the next evolution of the open web.  HMTL5 is currently planning on directly including it, so its support should increase dramatically over the coming months.  What we need is a better way of leveraging it.  sXBL is a start at componentization, but I think we need a more cohesive approach.

Starting from scratch would be extremely silly.  At the moment, I think that webkit has a very good base. It’s been proposed half-jokingly before, but I am saying non-jokingly now – I think some form of webkit as a plugin could work.  That is for a later date, though…


I don’t really expect anyone to pour time into some crazy scheme with no code to show for it yet, but if are interested and want to help, check out the group.  When I scrape some code together I’ll start a google code project.

Why HTML5 is not enough

I wanted to begin this post first by applauding the WHATWG/HTML5 crew for their work. They have been taking up the flag, fighting for the open web in their own way since 2004. You can read all about what is going on at the WHATWG site.

Let’s start with a pity party

I’m not being sarcastic. It sucks. I even feel bad for the internet explorer team. There are billions of web pages out there, and going forward, browser makers are in a position where it is difficult to move forward without breaking backwards compatibility.  A highly controversial posting by Joel Spolsky does a good job of at least describing the problem, even if you come to a different conclusion as he does. The WHATWG group took it as their goal to listen to web developers and take the step forward that we’ve been waiting almost 10 years for.

Here are some of the challenges they face in making HTML better for web applications:

  • HTML was and is a document format first and foremost.  In the document space, there is no challenge to HTML.  It has found its way into all imaginable types of devices and software.  It is simple and ubiquitous, and must remain so. It is one of the only environments that attempts to fluidly scale between document and application. That’s a hard thing to do. Flex and Silverlight don’t even really try.
  • It is a goal of HTML to be simple to create, but it’s still complicated enough that people screw up and the web is filled with tag soup.  Application developers may shudder at the thought (I do), but the fact is that legacy documents are still important, and that the outpouring of tag soup will probably not stop now.  Instead of “breaking the web” the WHATWG is just trying to standardize how to deal with tag soup.  I think that’s one solution, and may be for the best, but it is not an ideal environment for application development.
  • In addition to the richer needs of web applications, HTML must be capable of serving easily digestible, and indexable content.  It has to be accessible to screen readers and search engines.  Additionally, it has to be international, able to be used just as easily with other languages as it is with English.  These are some high requirements.
  • The HTML spec cannot make assumptions about user agent capabilities beyond HTML itself.  That means that while it can be developed with CSS and JavaScript in mind, it cannot assume that they are supported.

So what does that mean?

To put it simply, HTML means a lot of things to a lot of people. I think that it may be the increased importance and variety of use, as opposed to the stagnancy of internet explorer that has largely frozen the web.  It is a victim of it’s own success. Quickly looking at statistics, we can see that the numbers have jumped from 248 million users to 1.3 billion, and that the percentage of users across the world has exploded.  Additionally, we spend a lot more time and money on the internet, and we access the internet from all different devices.  More businesses are dependent on enterprise web applications than ever before.  Now you want to change it?!

There is no room for radical change in HTML. That is why XHTML flopped, and XHTML2 has failed completely. Only through more gradual and pragmatic evolution can the standards change. At least that is the thought process of the WHATWG. I won’t disagree – they are smart people.

Now what?

Well, WHATWG is getting us back on track.  Apple, Mozilla, and Opera are working together to get to this next stage at last. Even internet explorer is joining in.  So we’re finally going back to improving the internet, awesome. HTML5 is going really. Let me just add in that awesome new component I built:

<AwesomeControl id="myControl"/>

Oh right, no custom components. Well at least I can probably do some data binding so I don’t have to to all that wiring myself:

<input type="text" value="{inputValue}"/>

No?  Well, the competition can.  Don’t get me wrong, there are some things to look forward to – built in SVG and canvas, access to the browser history, client-side database, and even some nice form upgrades.  But let’s get a little perspective.  Real, large scale, rich client application development using HTML has been in the stone age for a long time.  This new stuff is great, but look at the competition.  Not only that, but HTML5 is still in draft form.  How long before it’s widely enough supported to actually be used? Flex and Silverlight are here now. JavaFX is on the way.


Dude.  So if HTML isn’t going to do the job (shouldn’t do the job), what is? Hopefully something that isn’t proprietary :)  For my part, I’m focusing on something new to compliment HTML, not replace it.  For the sake of speedyness, it’s going to have to be a plugin. If Adobe can do it, why not the open source community.  There are 1.3 billion users out there.  I reckon if you get the right 0.000001% and a bunch of open source libraries to start you off, it should actually be pretty easy, right ;)

Posted in propaganda. Tags: . 5 Comments »

Let’s Get Crackin’!

I recently had a conversation with Brad Neuberg about the concept of using a plugin to have an Open Web competitor.  Brad suggested that this was precisely what Google Gears was trying to do (sort of).  In a recent post of his (which has since sparked a conversation across the blogosphere), Brad discussed the definition of the term Open Web and its importance, but also how Gears can help to push the web forward. In our conversation, he asked, “If you were to add functionality to Gears that doesn’t enhance the web’s existing technologies, but rather creates new ones that live in the browser through Gears what would these look like?” The following was my response:


1. I think data binding needs to be built in – with mechanisms for formatting and validation (and if you mention XBL, I’ll respond with a blank stare.  Seriously? Stuff like that is the reason Flex and Silverlight are looking so good right now.)

2. HTML5 and SVG can get us most of the way there when it comes to visuals, but its not very friendly for application development because there’s no abstraction layer.  There is no way to compose or componentize a set of elements and attibutes together and then use it as a custom tag for example.  JavaScript frameworks have gone a long way towards simulating this, but it needs to be built in, and it needs to be easy. I would propose an addition to the HTML syntax that basically allows for templating in a very component oriented way.  These templates would be incorporated into the binding syntax, so that they could “re-template” when needed without having to make explicit DHTML calls. I would also advocate for staying with xml syntax.

3. CSS is a really cool idea, but it needs some re-evaluation.  First of all, it needs better layout management.  The hoops required currently are a little obscene.  Even adding a “layout-manager” property with a few possibilities would go a long way.  I would also definitely add CSS variables that, again, could be integrated with the binding layer.  CSS expressions were definitely a mistake, but variables would be extremely powerful.  For some ideas, just look at what webkit is doing with their css transformations and animations.  It would be so much simpler to just put a variable in.  Then a simple easing library could be used to change the variable over time to create animations. Finally, I think that CSS could be better incorporated into the new component model.  For example, it would be helpful to be able to scope rules to components, and allow custom tags to be selected.

4. I’m in the group of people that isn’t totally gung ho about JS2.  I understand the motivation, but its looking like a bit of a kitchen sink language. I think JS needs some improvements, but I’m actually looking at Erlang for inspiration instead of Java and Python.  In my vision of a future JavaScript, I see a few things.  First of all, I think there are some functional language features that would be good to add considering JavaScript is a lready a very functional language. I would like to add overloaded functions that use matching and guards for differentiation. I would also like to steal some aspects of the big Erlang feature of concurrent processes.  Here, I think, is a perfect convergence with Gears.  The Gears worker pools are a lot like Erlang processes (which I’m guessing you knew).  No shared state, separate process, and no access to the dom.  As I’m sure you know, this can be extremely helpful when trying to stay secure doing mashups, offload intesive operations from the main thread, and communicate with the server. Additionally, I think that there needs to be a little more in the way of modularizing code, allowing private data members, and facilitating better code reuse.  I think prototypal inheritance and mixins are definitely better than classes for the language, and I’d like to add some more syntax to encourage them.

It’s great to talk about the deficiencies in the technologies we have, but to really move forward, we need a conversation for what can come next. And by next I guess I mean the next next.  HTML5, CSS3, and JS2 are all really great things.  I encourage people to be as involved in those as possible, but I still have a lot of doubts about its ability to stay pragmatic, on track, and happen quickly.

If you could snap your fingers and have a plugin with a wide install base built for web applications (as opposed to web sites), what would it look like?  You can leave a comment, or even better, take a look at the group.

The Premise

If you haven’t seen it already, I recommend you watch this presentation by Douglas Crockford. Douglas Crockford, love him or hate him, knows a lot about internet technologies.  Even if there are aspects you do not agree with, I think his presentation makes many good points. To summarize:

  • Despite its flaws, standard web technologies (html/css/javascript) are hugely successful and have been able to accomplish what others have not.
  • It provides a taste of the potential for the internet – mashups, social computing, powerful web-based applications.
  • We’ve used (abused) every ounce of functionality made available by browsers, but we’re reaching the end of line.
  • There has been very little advancement of the browsers in the past 10 years.
  • We need to move forward, but there is no clear path.

Every web developer is aware of the difficulties of cross-browser ajax application development.  Javascript libraries have made a valiant effort and taken us a long way, but we have a long way to go.

New Developments

The competition for beating the new Acid 3 test has been fierce, and now Opera and Webkit have cooked up versions capable of passing. There is also  building support for the unfinished HTML 5 specification in several browsers.  Also, IE8 beta is passing Acid 2, as well as contributing some of its own new developments.  One might think that this is the answer.  Maybe we don’t have to worry as much as Douglas Crockford might think we do.  Here’s the problem: a lot of people are still using IE6!  Even if all browsers came together right now – even if IE replaced their rendering engine with webkit, there would still be problems.  The web just doesn’t move that fast.

Other Thoughts

recent post by Chris Keene and Alex Russell describes this proprietary side of the web, the “unweb”. They propose that the way to save the open web is as follows:

The key to the web’s future is real competition between the browser vendors that will force them to evolve the browser quickly. These features include:

  • Auto update capabilities
  • 3-d rendering
  • Support for new semantics in HTML
  • In short, give us native ability within the browser to do what we otherwise have to do in Javascript libraries

I think that this is one possible route, and good advice in general.  But is it a realistic possibility? Competition between browsers is a good way of moving forward, but also leads to incompatibilities.  Javascript libraries will once again be the stopgap to manage these incompatibilities, with the end result being the lowest common denominator.  Definitely better than nothing, but the real question, is it better than the “unweb” alternative.

Possible Solution

I believe there is an alternative solution.  I believe it is possible to fight fire with fire. If Flex/Silverlight/JavaFX threaten the open web, is there a way to compete on the same playing field? If the w3c technologies can’t compete, can we take a different route? I propose that one very real solution to the problem would be to create an open source plugin technology to compete. It would allow a few things. First of all, it could ignore backwards compatibility because there would be nothing to be compatible to. Secondly, the cross browser issue would be resolved by being a plugin instead of a single browser implementation.

Posted in propaganda. Tags: . 3 Comments »