If you are starting a new project, I would recommend that you think about some of the ideas I am laying out below.

Graphic Design:

Apps will have different characteristics than webapps.  You generally will not just want to layer css ontop of your page, and consider it mobile enabled and done.

You will have to design a completely different set of views for your mobile app than your desktop site, and possibly even your tablet as each platform will offer different capabilities that you’ll want to take advantage off.  Your layouts will accomodate different styles, and furthermore different functionality.

Redirects: Another thing to consider is that an App doesn’t allow navigation off of the page, so you should think about that and what it means for your app.  You wouldn’t want to redirect your users off of your domain.

Apps should be targeted / focused towards doing one thing well, and bubbling that information up to the user.

Popups: These are a web browser concept, that doesn’t translate into the App experience, so expect that popups won’t work.

So if you want a kick ass App experience,  plan on having completely different views for different device sizes.  The good news is everyone will be writing the same language, and basically using the same stylesheets !

There must be a ton of user UI tips, for writing kick ass mobile /tablet  apps, if you have any add it to the conversation

Infrastructure:

WebApps probably won’t make too many changes to the internal infrastructure of data centers.  But to the infrastructure / mechanisms you use for exposing your product to the world, those are / have changed.  The rest api revolution has been replaced by the Javascript Object revolution.  Exposing your services as part of a Javascript SDK is the new kid on the block, and it’s knocking down walls.

So far I haven’t seen too much change in this area, the cloud is pretty much a given at this point, at least for a startup.

Personnel:

The time for the Android / IoS developer is over, don’t do it, don’t fragment your infrastructure, your support, your ability to ship early and often.. It’s just not worth it.  If you are a startup and anyone is telling you to go native, i would seriously get second / third opinions.

Having said that using Phonegap or something of the like is a good idea to help bridge some of the functionality gaps.

If you want to take a look at how to  load client side / server site javascript in phonegap and all have it process you should take a look at github.com/mozilla/soup

Why:

The advantage of being small is that you can move quickly.  Try doing that with an android release , iphone release, and website release.. All with differing capabilities / code bases, and then dealing with android fragmentation.  It will make you slow, and slow is bad….

Learn More.

Can be a daunting task, all this mozilla code, all these strange words, mochitest, mozilla-central, jumping into the code base.. All are daunting tasks, but hopefully this tutorial will walk you through the general process.

Learn by Example:

The example that I am going to show is the soon to be performed work on mozilla central to port the webapps extension into the mozilla code base.

The exciting part about this we get to test against code that hasn’t been written.  The codebase i want to test will eventually be on mozilla central , but is currently not.  I didn’t select the option of testing against a debug build, because we might actually want to test against patches to the codebase. (which is what we are doing)

Building Firefox:

https://developer.mozilla.org/En/Developer_Guide/Build_Instructions

MochiTest Basics:

https://developer.mozilla.org/en/Mochitest

Assuming you have built mozilla central and you are able to run MochiTests, then what next.Well firstly you will want to choose a location for your tests.

Now If you are working on webapps this directory might be: dom/tests/mochitest, as it looks as a lot of tests that are dom related lie there. For each project it will be different, and you may want to seek out different opinions as to where things might go.

You will then have to create a directory “webapps”, and edit Makefile.in and add your directory below

DIRS  += \
dom-level0 \
dom-level1-core \
dom-level2-core \
dom-level2-html \
webapps \
storageevent \
$(NULL)

Next step would be to create a Makefile.in inside your directory foobar.  Below is a github commit by Geo Mealer which establishes our base repository.

Github Webapps MochiTest Repo

Running your Dummy Test Case:

navigate to the top directory in your mozilla-central codebase

TEST_PATH=dom/tests/mochitest/webapps/test_dummy.html make -C obj-ff-dbg/ mochitest-plain

Level II on your path:

Next you will be forced to consider / raise several issues with your development team.

You will want the ability to pref off certain chrome elements.

• prefs.setBoolPref(“apps.testing.mode”, true);
• prefs.setCharPref(“apps.testing.allowmgmt”, “http://mochi.test:8088″);

Determining what those prefs are that accomplish the goal is a conversation that needs to happen with the development team.!

It’s best to hold off writing until you have a general agreement, about this piece.

•  Designing your test framework.

Mochitests are a simple html / js test framework, it uses simple expressions like “is”, “is_not”, “ok”.  So the real difficulty is in designing your test framework to get the results you need.  Sometimes you will need greater access to objects other than prefs, sometimes you will need access to specific data structures to retrieve and set information.

There are tons of examples in the mochitest framework to work with:

localStorage, geolocation, general all have good examples. It really depends on what you are attempting to do, and how much effort you want to expend.

• Webapps Test Run

The test fails because the code isn’t implemented.  This codebase is a part of the same repo, but in a branch entire suite btw TDD.

What we have gotten done so far

• Basic set of tests working in Windows / Mac
• A working documentation set
• We have a few people contributing so far

I would like to say thank you to a few people for their support so far:

• Mohamed Dabbagah our intern from last semester, who has been pitching in whenever he can
• BYK – https://github.com/BYK – Thanks for the python tips!!

Please remember to not forget to note the Sikuli Test Day will be January 20th 2012

If you have spare cycles, and are interested in the project, check us out on github

Pick up an issue, try to solve, ask questions, contact me.. whichever suites your fancy.

-David Clarke

twitter: onecyrenus

email:dclarke@mozilla.com

This is a project that is meant to test the Web Apps Runtime for Mozilla.

https://github.com/mozilla/openwebapps/

https://github.com/dclarke/apps-extension.sikuli

If there are any python reviewers out there, it would be great to get some feedback / pull requests if you see anything horribly offensive !

I am interested in learning more about python, so headed to pycon 2012 this year. !

https://us.pycon.org/2012/

As with any new tool there are ups and downs, installation difficulties, learning curve..etc.. The hope with this test day is that you can walk away with expertise / know how , how you can roll your own vision based testing framework, and solve your own problems either in your day to day computing, student, or professional lives.

My belief is that some of this technology has the possibility to be transformational, and there is a huge capability here to make automation simple, or at least simpler .

This Test Day is for you if you are interested in any of the following.

Connect to the mozilla community. Become a Mozillian, and shape the future of the web!
Learn about Web Applications !
Learn a new Test Automation Strategy that is current, and applicable !

To learn more about this test day please and how you can contribute just read:
https://quality.mozilla.org/2012/01/webapps-extension-based-testing-with-sikuli-january-20th-2012/

If you are new to test days, please read our FAQ
https://quality.mozilla.org/docs/testdays/testdays-faq/

-David Clarke
onecyrenus

We had a great day of testing & finding bugs & engaging all of the core teams from UX –> Product –> Dev –> QA.

We got everyone involved, testing web apps and having fun.

Here are some stats from the day:

• 32 filed / modified bugs across multiple products
• 18 Commits https://github.com/mozilla/openwebapps/
• 4 Commits to Soup
• At times over 60 people in #testday
• 6-7 Contributors from Indonesia showed up at midnight PST

Operationally we were able to accomplish the following:

• We tested across multiple revisions of the extension / soup app
• We continuously pushed changes to myapps.mozillalabs
• We fixed / verified bugs across Windows / Linux / Mac OS X

Another component to the test day was to gather usability feedback from our UX team. And we were able to directly gather that information, and feed it back to our development team. So look for some usability improvements to come!

If you are interested in testing web apps I would recommend signing up to this blog for updates on the next test day, as well as following @MozillaQA on twitter.

Through the testday ether pad linked below you can still grab development builds and continue testing && submitting feedback. New builds are pushed out as soon as they are available!

Links:
https://etherpad.mozilla.org/testday-111202-webapps
https://etherpad.mozilla.org/AppsTestDay

Generate 1000′s of tests
Factoids

Lots of arrows – Lots of tests
precisely hit bulls eye – degree of closeness
cost of missing – special tests
how does it move – monitoring
unexpected movement – spike
Precision + Accuracy – Non- determinism

– Understanding your production environment, structuring your tests / test environments are key concepts in any test plan. Just testing positive scenarios, or black box testing is not enough.

Testing Cloud Failover
- Walk over failure paths
- Multi-cloud approach

– This is ignored in many cloud startups, but is crucial if you want to run a cloud. Companies too often start at the product, and work their way down only to realize several outages later that their core switching network doesn’t allow for failover, or their db schema doesn’t lend itself to be sharded.

ABFT in the Cloud
Automated Basic Functionality Tests
What is in the cloud ? Shipping platforms
Visual Test – uI Workflow testing ( similar to visual basic)
14 hr regression run
1 + 1/2 GB of data from each run

– If you store it, you better figure out a way to expire the data or at least decrease data points over time, similar to rrd logs

This is the mission if you choose to accept it.

We are the Open WebApps project, and we want as many people looking over our javascript as possible.
If you imagine to write HTML5 / Javascript in the future this might be a great little intro to our API’s and how things work.

Below is a podcast on how to add unit tests:

Your mission is to find as many test cases / edge cases that are not represented in our test suite and get your submissions in to me.
Show me your pull requests in the comment feed and we can get this competition going!

*** Technical Details ***

Basically the test infrastructure is run from the tests described in: site/tests/*.html, and the javascript is in jsapi/*.js.

To the top contributor i will send a goodie bag of the coolest Mozilla stuff I can find. The coolness of the stuff depends on the coolness of the submissions!!!

#1) Javascript in CI, inside a browser is not a slam dunk as of yet. There are plenty of things to worry about so you want to be flexible in your solution.

We had initially taken a look at jstestdriver, but some things were not as great as we would have liked.  It is more suited for unit tests.

The concept is that the browser lives on the same machine, generally we want to have a broad array off browsers and the developer may not want to install all of that on their laptop.

#2) Technologies

We have tests written in doctestjs, a javascript test framework that was written by Ian Bicking, as well as some test cases written in Qunit.  At that time there wasn’t a test framework that could handle multiple formats, and report up the results.

This is what we have done in essence is to add the functionality to jstestnet / doctestjs to be able to collect reports from multiple test runners.  Currently qunit / doctestjs and report up the results.  There is an abstraction between the test runner, and the test reporter which allows us to create this environment. Exciting Stuff.

Below is a link to my vimeo screen cast of the end to end jenkins integration.  Please stay tuned as I will be shortly blog posting on how to add tests to the integration run. This will be more focused towards developers.

Let me know if you need more detail from the QA side, or if you are interested in writing unit tests!!

ScreenCast: OpenWebApps CI