Originally posted on JAWS:
One of the less covered parts of this week’s Firefox release is high attention that was placed on the performance of the redesign of the tab shape.
The new Firefox introduces a new tab shape that is consistent with Firefox for Android, FirefoxOS, Thunderbird, as well as the web properties of Mozilla.
As the Firefox team was implementing this new design, performance was a key metric that was measured and focused on. We wanted to not only bring a beautiful design to users, but one that matched the new sleek shape with an equally speedy outcome.
Each time a change was made to our source control repository, a fresh build of the browser was created and run against a suite of automated tests that measure the performance of the build. These results are then compared against the results of prior builds, allowing the team to track…
View original 297 more words
On the way back from the Mozilla Summit 2013, I spent the time to read Malcolm Gladwells new book. David and Goliath, underdogs, misfits and the art of battling giants, and I was immediately struck by the similarity between Mozilla and the story of the underdog.
The history of the browser has been one of constant change and innovation. Mozilla has been a company that has been the underdog, so much so that I don’t believe that anyone would have bet on the success of the Mozilla project at the very beginning (except it’s core actors). But 10+ years later the project is still going strong, albeit with new challenges, and new Goliaths to tackle in the industry.
The core issue that I feel we struggle with is in communicating our tremendous core value to our users, and similarly delivering a product that our community feel is a great world class / stable product for them to use.
Throughout the course of Mozilla Summit 2013 a question came up throughout the conference was as to why the state of the web so little resembles the web that we as users want. The question came up several times throughout the conference..
What is the type of web that users need ?
Is there a message or thought that will be appreciated, digested, and acted upon to make the end user more cognizant of the lack of choice. If users can understand that we are just in a more cleverly designed box than the Aol / Compuserve / Prodigy would they care ?
Each app ecosystem has it’s own gotchas, license agreements and restrictions which limit the inclusion and fraternity to which the web was designed. I remember our first experiments building web pages in high school. Changing font sizes, the blink tag, tables and the like. We all got to put up and manage our home page. A system that allowed for play, and learning for all, and was free.
If a student in school wants to make an iOS app you have to:
- Have a copy of Xcode (ergo a Mac OSX computer)
- Apply for a developer license ($99 / year)
- Have an iphone / ipad.
For students that don’t have access to such a technology stack this must be an insurmountable hurdle. If the web is not relevant to students, and their only point of entry is via their phones / tablets. What does this mean for internet literacy in general. Are we turning people away from the internet by not having a cheap / fun alternative, which allows people to experiment, build experiences and play.
Can Mozilla find itself in a nuanced position which will allow all users to play, and learn. Or will we live in a world where users don’t get that fundamental education. How will Mozilla do battle against it’s new Goliath?
Malcolm Gladwell notes that in a competition between a smaller adversary and a larger, if the smaller adversary takes a conventional approach to the confrontation the odds are the smaller adversary will loose, but in the situation where the smaller adversary takes an unconventional approach the probability will be that the smaller adversary will win.
By having the advantage of agility, a scrappy attitude and the will to change the internet, “users” could find themselves again in a winning position, where they are once again able to play and learn in an environment that is built for that very purpose…
Gathering profiling Data on Firefox OS can be done in several different ways.
#1) Simpler is better, sometimes a stop watch is a great medium with which to gather timing data for an application you don’t understand.
2) In Application Profiling
var d = new Date();
console.log(“The time since epoch = ” + d.getTime());
Once you insert a few of these statements in “strategically” placed points.
Just cd gaia; make install-gaia, this will install the new version of gaia to your phone, and you can see what code path gets triggered at which points.
3)Measuring app startup Times
Turn on show app load times by loading your settings app.
Click Device Information –> More Information –> Developer –> Show Time to Load.
This is one mechanism for measuring app start times, but this only measure time until mozbrowserfirstpaint which measure the time until first pixels are available for an app.
More details can be found in Bug: 787378
Performance testing (http://en.wikipedia.org/wiki/Software_performance_testing) for B2G is being done by many dogfooders / early users as we speak. The beginnings of this effort are to begin a framework for gathering this data, as well as empower both power users and employees to really start to make sense of all this complex data !.
Toolset: Mind, stop watch, performance profiler, about:memory tools
First Steps in performance measurements, were hashed out over the B2G work week most recently in San Francisco. The most immediate things we want to gather data on are the following.
- B2g Startup Time:
Measure / Record the startup time for the overall system. We want to know how long it takes to get to the unlock screen.
- Use case: As a user I do not want to wait for a long period of time before my phone will start.
– The phone is off
– Start phone
— wait for unlock screen, with animation.
- B2G Startup Time for each App:
- Use case: As a user I want to be able to open apps one after another and have the operating system be able to quickly load each app, and switch to the homescreen.
Part II of this blog post will go over How To’s of memory gathering / dissecting.
Tune into Part II: About Memory Gathering / Dissecting
As we progress through the webapps project there is an astounding see of change coming to HTML5 Apps near you.
If you are starting a new project, I would recommend that you think about some of the ideas I am laying out below.
Apps will have different characteristics than webapps. You generally will not just want to layer css ontop of your page, and consider it mobile enabled and done.
You will have to design a completely different set of views for your mobile app than your desktop site, and possibly even your tablet as each platform will offer different capabilities that you’ll want to take advantage off. Your layouts will accomodate different styles, and furthermore different functionality.
Redirects: Another thing to consider is that an App doesn’t allow navigation off of the page, so you should think about that and what it means for your app. You wouldn’t want to redirect your users off of your domain.
Apps should be targeted / focused towards doing one thing well, and bubbling that information up to the user.
Popups: These are a web browser concept, that doesn’t translate into the App experience, so expect that popups won’t work.
So if you want a kick ass App experience, plan on having completely different views for different device sizes. The good news is everyone will be writing the same language, and basically using the same stylesheets !
There must be a ton of user UI tips, for writing kick ass mobile /tablet apps, if you have any add it to the conversation
So far I haven’t seen too much change in this area, the cloud is pretty much a given at this point, at least for a startup.
The time for the Android / IoS developer is over, don’t do it, don’t fragment your infrastructure, your support, your ability to ship early and often.. It’s just not worth it. If you are a startup and anyone is telling you to go native, i would seriously get second / third opinions.
Having said that using Phonegap or something of the like is a good idea to help bridge some of the functionality gaps.
The advantage of being small is that you can move quickly. Try doing that with an android release , iphone release, and website release.. All with differing capabilities / code bases, and then dealing with android fragmentation. It will make you slow, and slow is bad….
Just thought I’d spend a few minutes and blog about the Web Apps Sikuli Project.
What we have gotten done so far
- Basic set of tests working in Windows / Mac
- A working documentation set
- We have a few people contributing so far
I would like to say thank you to a few people for their support so far:
- Mohamed Dabbagah our intern from last semester, who has been pitching in whenever he can
- BYK – https://github.com/BYK – Thanks for the python tips!!
Please remember to not forget to note the Sikuli Test Day will be January 20th 2012
If you have spare cycles, and are interested in the project, check us out on github
Pick up an issue, try to solve, ask questions, contact me.. whichever suites your fancy.
Recently just started my first open source project, hosting on github, codebase is in Python.
This is a project that is meant to test the Web Apps Runtime for Mozilla.
If there are any python reviewers out there, it would be great to get some feedback / pull requests if you see anything horribly offensive !
I am interested in learning more about python, so headed to pycon 2012 this year. !
HTML5 Web Applications are coming to a browser near you. This past friday(12/02) we had a great test day at Mozilla focusing on testing web applications.
We had a great day of testing & finding bugs & engaging all of the core teams from UX –> Product –> Dev –> QA.
We got everyone involved, testing web apps and having fun.
Here are some stats from the day:
• 32 filed / modified bugs across multiple products
• 18 Commits https://github.com/mozilla/openwebapps/
• 4 Commits to Soup
• At times over 60 people in #testday
• 6-7 Contributors from Indonesia showed up at midnight PST
Operationally we were able to accomplish the following:
• We tested across multiple revisions of the extension / soup app
• We continuously pushed changes to myapps.mozillalabs
• We fixed / verified bugs across Windows / Linux / Mac OS X
Another component to the test day was to gather usability feedback from our UX team. And we were able to directly gather that information, and feed it back to our development team. So look for some usability improvements to come!
If you are interested in testing web apps I would recommend signing up to this blog for updates on the next test day, as well as following @MozillaQA on twitter.
Through the testday ether pad linked below you can still grab development builds and continue testing && submitting feedback. New builds are pushed out as soon as they are available!
How to hit a moving target ?
Translation Service for Microsoft
Log Aggregation Service
-Logs / Trace / Statistics
- Data Available for Querying
- Challenges Personally identifiable info
Generate 1000′s of tests
Lots of arrows – Lots of tests
precisely hit bulls eye – degree of closeness
cost of missing – special tests
how does it move – monitoring
unexpected movement – spike
Precision + Accuracy – Non- determinism
– Understanding your production environment, structuring your tests / test environments are key concepts in any test plan. Just testing positive scenarios, or black box testing is not enough.
Testing Cloud Failover
- Walk over failure paths
- Multi-cloud approach
– This is ignored in many cloud startups, but is crucial if you want to run a cloud. Companies too often start at the product, and work their way down only to realize several outages later that their core switching network doesn’t allow for failover, or their db schema doesn’t lend itself to be sharded.
ABFT in the Cloud
Automated Basic Functionality Tests
What is in the cloud ? Shipping platforms
Visual Test – uI Workflow testing ( similar to visual basic)
14 hr regression run
1 + 1/2 GB of data from each run
– If you store it, you better figure out a way to expire the data or at least decrease data points over time, similar to rrd logs