Making Facebook's native mobile app faster using HTML5(sencha.com)
When we started what became Sencha, we made a bet on the web: a bet
that modern application development didn't need anything except the
browser, a great set of frameworks and a great set of tools. With those
three weapons in hand, we knew developers could build applications that
would delight users. The advent of HTML5 upped the game and it gave
developers even more tools to let them treat the browser as an
application development platform and not a page rendering engine.
Developers sprang at the opportunity and unleashed a torrent of apps —
on both desktop and mobile — that leveraged the new HTML5 capabilities
to build amazing applications using web standards.
So, when Mark Zuckerberg said HTML5 wasn't ready, we took a little offense to the comment.
We thought to ourselves: HTML5 can't really be the reason that
Facebook's mobile application was slow. We knew what the browser on
modern smart phones was capable of and what kind of rich capabilities
HTML5 offered. We saw the latest generation of mobile devices — running
at least iOS 5 or Android 4.1 — push ever increasing performance and
HTML5 implementation scores. But perhaps most importantly, we'd seen
what our customers were building and the amazing things they were
creating using HTML5.
“So, when Mark Zuckerberg said HTML5 wasn't ready, we took a little offense to the comment.”
We had our suspicions about why Facebook's mobile application team had problems,
because it matched a common pattern. At Sencha, we build frameworks and
tools for application developers, so we have pretty deep experience
with development teams taking on HTML5 app projects. When a team has
problems with HTML5, it usually stems from the fact that they take a
“website” development approach to building an app, and often don't use
the right tools and architectures for application development. This is
what we suspected about the Facebook HTML5 app. The way that app
performed — slow loading, choppy user experience in the News Feed, low
framerate — exhibited the usual symptoms.
In any event, we knew HTML5 was, in fact, ready, and we wanted to
prove it. So we took it upon ourselves to rebuild the challenging parts
of the Facebook mobile application in HTML5 in our spare time. Today,
we'd like to introduce you to Sencha Fastbook,
a technology proof of concept that shows how fast HTML5 can be, and
demonstrates how readily HTML5 can be used to handle the toughest app
challenges.
Learn the latest about Fastbook and the tech behind it at SenchaCon 2013. Register today!
This four-minute video gives you a quick overview of Sencha Fastbook,
and shows you a side-by-side comparison of how well our HTML5 app
performs against both the native iOS and the native Android Facebook
apps (versions 5.2 and 1.9.12 respectively, the latest available when we
made this video on December 10th). The rest of this post gets into the
technical details of how we built Fastbook.
A Closer Look at the “Native" Facebook Application
We started the process of what became Fastbook by trying to
understand the Zuckerberg claim that HTML5 “just wasn't there”, and the
best place to do that was to take a deep look into the latest native
Facebook app on iOS. We connected an iPhone to a web debugging proxy,
and started to look at the HTTP traffic the application pushed over the
network. Our biggest shock: much of the application was still raw HTML
pages. The News Feed had moved to native as had the profile page, but
many of the other application UIs were simply HTTP GET requests to
m.facebook.com. Today's “native” Facebook application is a hybrid web /
native application: there is content rendered on m.facebook.com and
displayed in a UIWebView and native Objective C components mixed
together.
Re-Implementing the News Feed
After we looked at how the native Facebook application worked, it was
clear to us that the hardest part of the experience to build was the
News Feed. Dealing with a billion content creators posting an unlimited
amount of content in a completely unpredictable manner is a tough
problem to solve even for the most seasoned developers, regardless of
what technology they use.
We really wanted to ensure we had that smooth experience the News
Feed should deliver when we re-implemented it in HTML5. To make that
possible we added some new features and enhancements into the core of
the current Sencha Touch framework.
It started with the implementation of an Infinite List Component that
handles items with unknown sizes. Only a very small set of DOM nodes is
actually created to fill in the actual visible screen area. They will
then be constantly recycled to render next / previous data on-demand. As
a result, memory footprint is kept minimal, regardless of the amount of
data in the Store. Making this work is the easy part. Making it fast
with the complexity and variety of items such as News Feed stories is
the real challenge. The bottleneck lies within the core processes that a
browser has to perform: layout and compositing.
Our experience with building frameworks has taught us that while a
small demo component might work well on its own, it often then performs
poorly when put into a much larger app. As an app grows, the DOM tree
grows; and as the DOM tree grows it takes longer for the browser to
calculate the layout, and the performance degrades. Moreover, as the
number of visible layers increase, compositing performance for each
layer also degrades dramatically. We needed a solution to make web apps
more robust under large numbers of DOM nodes.
We do a side-by-side comparison of the native Facebook app and our Sencha Fastbook app.
So the Fastbook app is the first to make use of a brand new “Sandbox
Container” which programmatically detaches complex views and renders
them into their own iframes, and thus partitioning the DOM tree. This
special container doesn't need any extra handling at the application
level, so it's seamless to developers (i.e., any component added to this
container will be sandboxed automatically). But it does come at a cost:
events, positioning, styling, and JavaScript code have to be proxied
between the parent window and the child sandboxes. This is complicated,
so without a robust and properly architected framework, it is very
difficult to implement. Sandboxing allows layouts to be isolated, and
therefore keeps the primary DOM tree as light as possible. To bring
balance to the Force, Sandbox Containers must be used wisely.
For Fastbook, the News Feed, Timeline and Story views are
individually sandboxed. Since all DOM elements are re-used to render
data on-demand at high frequency, reflows are inevitable. The key is to
make that process as cheap as possible. Sandboxing enables the News Feed
to perform as if it is standalone, while actually is still a part of
the much bigger DOM tree.
“Sandboxing enables the News Feed to perform as if it is standalone, while actually is still a part of the much bigger DOM tree.”
Next, we added deeper integration to our TaskQueue, a feature we
recently introduced into Sencha Touch. TaskQueue prevents the
interleaving of read and write requests to the DOM, eliminating any
unnecessary layouts. This, combined with the new sandboxing technique,
significantly reduces costly layouts from complex views such as the
Timeline and News Feed.
We then added the AnimationQueue, a new class that's responsible for
all animations and events, as well as scheduling heavy tasks for later
execution during the CPU's idle time. It acts as the framework's traffic
cop, prioritizing different operations and helps ensure the application
stays responsive. When an app is animating, it suspends lower priority
functions. When the app is idle, the AnimationQueue executes the
suspended tasks. For example, while scrolling the News Feed at high
speed, image loading and rendering is suspended until the app is idle in
order to increase the scrolling performance. Additionally, heavy tasks
are released gradually in a non-blocking manner with a high-speed timer.
This ensures touch events always have a chance to be handled as soon as
possible.
On the other hand, there are some classes of functionality that you
don't want to suspend, such as getting more data to feed the list. To
help ensure this doesn't slow down scrolling, we use Web Workers. These
allow us to move XHR/RPC communications away from the UI thread. Saving
network request cost and JSON encoding/decoding using Web Workers makes great use of today's multi-core devices.
Those are the major points of what we did to bring Fastbook to life,
and what allows it to perform well with pure open-standards based web
technologies. It's exciting for us to be able to show how you can
leverage all the features of HTML5 to build an app like this.
Bonus Points
One of the interesting things we found while investigating network
performance of the native iOS Facebook app was that the API calls
returns huge amounts of raw data to the client. A typical example is API
calls made to https://graph.facebook.com/graphql/
to render News Feed items. In average, 15KB to 20KB of gzipped JSON
data is transferred for every 10 items, much of that is not needed to
render the actual views.
To demonstrate what could be done to optimize network transfers, we
put a proxy server in place to clean up and parse the raw data returned
from the Facebook FQL API. As a result, Fastbook transfers far less data
than the native app to render the same views: as little as 10% to
render the same items on the News Feed. The proxy also allows us to
offload some of the more mundane tasks such as content formatting and
filtering to the server-side.
Also, you may notice a difference in the scrolling deceleration time
between the native iOS app and Fastbook. In the native app, scrolling
doesn't stop for about 3s. We decided to increase friction and reduce
the animation duration to 1.4s. This not only makes content ready to
read faster, it also provides extra idle time for the app to buffer more
items while the user is still reading existing content.
Try It Yourself
Fastbook isn't a replacement for the Facebook application. It's a
technology demo that shows what developers can do with HTML5 if they
take the right approach, and use the right frameworks and tools. If
you've been wondering if HTML5 is ready, try Fastbook
for yourself on a modern smartphone (we recommend at least iOS 5 or
Android 4.1). You'll see that when you treat the browser as an
application platform and leverage the features of HTML5 that even the
most sophisticated applications can be made with HTML5.
I’ve started working on a few presentations to take on the road to
developer conferences and one of the topics is a presentation on HTML5.
Many folks look at new technologies from a somewhat academic
perspective, like folks that are frequent public speakers at technical
events. This is an important part of the technical evolutionary process.
Folks that push the edges of new technologies for their own sake lay
the groundwork for how will will apply those technologies in the future.
But I tend to like to focus more on the practical business applications of new technology.
We all know that the World Wide Web is broken.
Once upon a time there was the idea of the World Wide Web as one huge semantic network of information.
se·man·tic
adjective 1. of, pertaining to, or arising from the different meanings of words or other symbols
What we have instead is a glut of content.
From my perspective, one of the most significant enhancements coming in HTML is the collection of additions to semantic markup.
A huge problem with HTML prior to version 5 is that markup lacked any real informational context.
Tags like DIV, SPAN, and TABLE are used to organize the presentation
of the content on our page and, of course their associated Id tags can
be used to execute code against those elements. Also we have a
collection of tags that we can use to describe the aesthetics of other
tags, like style, class, width, etc.
But, none of that markup helps us identify actually what is contained inside those tags.
The HTML5 <article> tag, as well as other “semantic” tags in HTML5 go a long way to enable us to solve this problem.
Here is what the HTML5 spec says about the <article> tag.
The article element represents a component of a page that
consists of a self-contained composition in a document, page,
application, or site and that is intended to be independently
distributable or reusable, e.g. in syndication. This could be a forum
post, a magazine or newspaper article, a blog entry, a user-submitted
comment, an interactive widget or gadget, or any other independent item
of content. W3C Specification
Take special note of the phrase “self contained composition”.
Lets look at how an article might have been presented in HTML 4.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<div>
<h1>The 2011 Super Open Source Event</h1>
<div>
<h2>Keynotes</h2>
<p>Important people talking about important stuff.</p>
</div>
<div>
<h2>Sessions</h2>
<p>Breakout sessions for learning technical details.</p>
</div>
<div>
<h2>Roundtables</h2>
<p>Open discussions about the new technology.</p>
</div>
<strong>Link to get all the videos after the event.</strong>
</div>
Though this content is well organized, there is no semantic context to the content.
So ? You may ask why that matters.
It might not matter if you only want to read content on sites that
you already know about and visit, but that’s really far less than the
way the web was meant to work.
In fact, even in today’s web, this use-case accounts for a VERY small
percentage of the way people actually use the web. The popularity of
search engines is clear evidence that people begin most of there
interaction with the web by starting with a discovery process. The
content above provides very little guidance to discovery mechanisms like
search engine indexers which primarily have to rely on word indexing
and complex inference algorithms to determine the nature of the content
on a page and, even at that, the search engine can’t really help you
consume the content it finds, it can only send you to that “page” for
further determination as to the appropriateness of the page for your
needs.
We have created complete additional technologies like RSS and ATOM to
help solve this problem. The problem with those technologies is that
they are additions to the actual content. We need to create the content
AND we need to create the “feed” that defines that content.
While this works, it is a compete secondary effort and it only works
for content when the feed data is explicitly created for that given
content. The rest of the web is still just a bunch of words.
Let’s say, for example, that I was building a web site for Tennis
fans. If I wanted to index and aggregate articles from various Tennis
web sites and those web sites used markup like that above, it would be
nearly impossible for an application to figure out what constituted a
story on that site.
But, using HTML5′s semantic markup, the content itself contains logical context.
Let’s look at the content above in HTML5 markup.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<article>
<h1>The 2011 Super Open Source Event</h1>
<section>
<h2>Keynotes</h2>
<p>Important people talking about important stuff.</p>
</section>
<section>
<h2>Sessions</h2>
<p>Breakout sessions for learning technical details.</p>
</section>
<section>
<h2>Roundtables</h2>
<p>Open discussions about the new technology.</p>
</section>
<strong>Link to get all the videos after the event.</strong>
</article>
You can also see an additional addition to HTML5′s new semantic markup –
the <section> tag which can be used to segment an article into
logical sub units.
The section element represents a generic section of a
document or application. A section, in this context, is a thematic
grouping of content, typically with a heading.
Examples of sections would be chapters, the various tabbed pages in a
tabbed dialog box, or the numbered sections of a thesis. A Web site’s
home page could be split into sections for an introduction, news items,
and contact information.
There's a lot of hype going on about html5 nowadays. If you're not
using html5 yet, the whole thing can be somewhat overwhelming. After
reading this blogpost you'll be ready to upgrade your current project
(or start a new one) using html5.
Understanding html5
I went to the bocaruby ruby users group earlier this month. I had some understanding of what html5 is capable of, but Diego Scataglini's presentation html5 a gentle introduction definitely expanded my knowledge. I recommend it as a solid introduction to what is different about html5.
Using html5
Often times after learning about a new technology a lot of
programmers get stuck. This new functionality is great, but how I do I
get started using it? It really helps to have something that gives you a
little push. enter html5-boilerplate
Its a frontend template written in html5 with a lot of goodness baked
in from the start. Between the css, html, various server configuration
files, javascript libraries and profiling, its a huge collection of sane
defaults for any web project.
Some of its many features
History API is quite a simple concept - a JavaScript API you can use
to control history state. If user clicks on an image and you show a
lightbox with enlarged version, clicking Back sends user
back to previous page, rather than closing lightbox popup. It does this,
because there is no state information browser can use to know how to
close that popup window.
With using History API you can add an entry to history stack once some dynamic content is loaded. When user clicks Back
browser would go back by one element in history stack and fire off an
event which then you can handle by closing the popup window. Same thing
happens when user clicks Forward - event happens and it’s up to your script to handle this gracefully.
All this behavior is very well explained in “Dive into HTML5”book,
but in short when you load some content using Ajax or you move user to a
place in a page which you want to have linkable - use History API. Of
course this involves handling the links in server side too, but this is
quite trivial - build the website first then make use of History API to
improve the interface. In my tests this improved user experienced a lot
and allowed to achieve very rich user interfaces without destroying
website structure.
Of course like with most of the new HTML5 functionality, this is not supported
in all browsers. Most importantly this is not supported at all in any
IE versions, not even IE9 which was released only couple months ago. But
you can handle this by having improved UI for some users and falling
back to normal non-js behavior for IE, for example this is what Github does (click on folders).
However I would recommend using some abstraction for this, mainly
because you need to do quite some manual work, which I hate doing. A
great tool exists called History.js which does exactly that - abstracts History API and also
has fall-back support for lame browsers like IE. I’ve used it
extensively and it works great and it even has adapters for jQuery et al
so you can use the same interface.
Most impressive part of it is optional fall-back for older browsers. What it does is uses a similar to hashbang url, like http://webspecies.co.uk/#/about which it handles inside and all methods for getting current url still return /about.
And if you go to this url from a modern browser it detects the support
for proper History API and redirects to correct url. All combined, makes
everything work nicely and future-proof, here is an example of full script for Ajax page.
How I used it
When building our Web Species
website designer had an idea to have all content in one scrollable page.
If you go to the homepage and start scrolling you’d notice that
immediately - menu stays in place but content slides scroll as usual.
Now even though this is very nice from user perspective, it creates a
problem from content side. Very big problem.
Main problem is that there is no way to link people to specific slide
and that it’s hard to get good rankings on Google. Now even though the
first problem can be easily fixed by using anchor urls (like http://webspecies.co.uk/#about),
but fixing SEO is, I believe, impossible. Impossible because one page
contains a lot of content and you won’t get good rankings for any
keywords.
If you look at view source of http://webspecies.co.uk/about
you’d see correct title and only “about” content, but as soon as you
load this in browser AJAX loads all slides and replaces existing slide
with them. So from user perspective there still exists one-page effect,
but actual pages can be called directly (in server side I have all
separate pages like about, training, clients etc. and one with all
slides).
To fix linking I employed History API and it worked out beautifully.
If you click a menu item on left side of the website, AJAX event fires
off resulting in two actions - pushState() and scroll to correct slide. Same thing happens on Back or forward history actions - content is scrolled to where it belongs. If you go to news section clicking on news items result in same behaviour too.
Conclusion
As with all HTML5 functionally History API is still quite rarely
used, but if you feel like building something awesome - I’d say give it a
go. It works beautifully in modern browsers and there are tools which
can make it happen for older browsers too. From my point of view, it
allows to achieve crazy UI ideas and still have semantically correct
websites, which is always my goal.
Author:
Juozas *"Joe"* Kaziukėnas, CEO of Web Species Ltd.
Can be reached using twitter as @juokaz or in his website http://juokaz.com.
In addition to HTML5, I will cover how to remove some of the
generated markup Drupal spits out by default, and how to override
default system CSS. I will also provide a few tips on how to get some quick-win page speed boosts.
Before we dive in, I just wanted to mention that this won’t be a comprehensive how-to on building a Drupal 7 theme. For the sake of brevity, I won’t be covering the HTML5 Tools
module, which endeavors to add features like new HTML5 form elements to
Drupal. Instead, I will focus on trimming the XHTML glut that Drupal
still instinctively wants to include, stemming from its earlier days.
Remember — back before the W3C declared XHTML 2.0 is dead,
heralding HTML5 as the future — when we all used to put type="text/css"
on tags, as if we’d ever used a <style> tag that wasn’t CSS? Oh how silly we were! Yet we thought we were so cool, appeasing the validator. Then HTML5 came along, paving cowpaths, exposing the tedium of so many practices we’d once thought necessary.
Structure
When you first install Drupal, you will see a top-level directory named themes. Common sense would tell you that this is where you should store your own theme related files. In this case, common sense is wrong! In actuality, the directory you want to play in is sites. This is all about you and your particular site(s), and therein is another themes directory that will house your theme(s).
Note: Drupal can handle multiple domains via a single codebase,
hence the “sites” directory instead of “site,” but that’s beyond the
scope of this article. If you’re interested, there’s a Multisite Drupal group dedicated to the topic.
The first thing you should do is create a *.info file with the name of your theme. Since my site is sonspring.com, and my theme is aptly named “SonSpring,” my info file is sonspring.info. Here are the contents (”;” comments out a line)…
If you’ve looked at any Drupal themes, the preceding code is pretty
straightforward stuff, nothing too out of the ordinary. One thing you
might have noticed is the override directory. Inside it are two other directories, kill and keep.
I learned this trick from Morten Birch Heide-jørgensen during one of his talks on Drupal theming. He calls his folder foad and the technique FOAD, short for “F*** Off And Die” – a clever way to get rid of system CSS files you don’t want.
If you name one of your own CSS files the same as one of the
defaults, Drupal will include yours instead of providing its own.
Thusly, everything in the kill directory is just an empty file, whereas everything in the keep directory is a CSS file that’s been only slightly (if at all) modified from its original defaults.
Now, you might be wondering: Why bother having a keep
directory at all, if I’m reusing the very same code as the default files
it contains? Here’s where it gets interesting. When you go to the Performance
admin page, and turn on aggregation and compression of CSS files,
Drupal will gather up all its default system stylesheets into a single
file, and all your stylesheets into another.
That’s silly. But with my keep directory, I force those default stylesheets – along with my theme stylesheets – to be aggregated and compressed into one file.
Template.php – Section
Call it an old habit from my beginner CMS days using Textpattern, but for theming purposes I like to know what “section” of the site I’m on. Basically, that just means I am curious what foobar is equal to, in the following scenarios…
So, at the beginning of my template.php file, I have a
simple function that runs through and grabs me the first path fragment
in the URL, does a simple switch/case, and returns a string that becomes
the <body> ID.
In my template.php file, I have a function called sonspring_process_html_tag. Obviously, if your theme is named “MyTheme” then your function would be named mytheme_process_html_tag. It simply unsets the variables for the type and media attributes on style, link, and script tags.
It also clears out CDATA comments, that only ever existed to satisfy
the XHTML validator, because hardly anyone ever served XHTML as
application/xhtml+xml. In the past, even though we all served XHTML as
text/html, we still jumped through hoops by adding CDATA comments. Not
so with HTML5.
In reality, XHTML always worked fine
without the needless code. HTML5 just makes it official. By way of
comparison, here’s the before and after of what Drupal would normally
output by default, followed by the slimmed down result. Before – XHTML
Though this doesn’t pertain to HTML5 directly, I decided that while I was mucking about in the template.php
file, I might as well add some HTML
minification. Basically, the following code just strips out unnecessary
whitespace and line breaks, saving a bit of file size where possible.
If a page contains a <pre> or <textarea> tag, it is left
alone, as to not affect code blocks or text typed in a form.
While it is technically possible to make use of HTML5 and RDF
together, I have opted to simply leave this aspect out of my templates,
as you will see shortly. As such, I have simply disabled the RDF module (enabled by default).
For the most part, Drupal only renders what’s in your templates. But
there are a few areas where it tends to wrap things in <div>
tags that you might not want. Thankfully, Drupal’s theme system makes
it easy to remedy. The following is a list of files that I have created,
which contain only the code necessary to generate your content, thereby overriding Drupal’s default output.
I have purposefully omitted the closing ?> tag, because it is considered a best practice (by both Drupal and Zend) to not close the <?php tag. Instead, the parser realizes the PHP
code is finished when it has reached the end of the file. This prevents
the unintentional injection of whitespace into a page. (Needless to
say, ?> is still necessary when mixing PHP and HTML within the same file.)
Note: The templates that start with “views-” pertain to Views, a third party Drupal module that makes it easy to output lists of content based on various criteria. block.tpl.php
Note: I have removed $item_attributes from my field.tpl.php
file. This is used by the RDF module, and can potentially be utilized
by third party modules, to add things like onclick="..." to various
elements. I removed it because I am not using the RDF module, but your
mileage may vary — read more.
Though the aforementioned files pertaining to block, field, region, and views are all technically templates, in that they end in *.tpl.php, the crux of the templating system revolves around HTML, page, and node templates.
Below, I have listed out the majority of my template code. You may notice I did not include my page‐‐front.tpl.php file. It is mostly hard coded because it does not change that often, with the only dynamic part being recent journal entries. html.tpl.php
For the sake of completeness, I figured I ought to finish by listing out the contents of my *.inc files, contained within an includes directory under assets. While these files could have just as easily been left in the higher level *.tpl.php
files, to ease in maintainability and in an effort to adhere to DRY
principles, I separated them out into their own files. By having the
file extension *.inc, Drupal knows to disallow direct browsing, so they can only be used via inclusion in a PHP page. header.inc
While all that might seem like a lot of code at first glance, keep in mind that’s pretty much the entirety of my site’s theme (aside from CSS
and light JavaScript). Once you get the hang of how it works, the
Drupal theming system really not that difficult to understand. Plus,
you’re learning real PHP, not pseudo code.
I wouldn’t say Drupal is perfect,
but it offers extensibility via simple overrides. By far, it is the most
intuitive approach I’ve found in my meandering quest for the ideal CMS.
Hopefully this walkthrough has helped dispel the myth that Drupal’s
learning curve is “steep,” or at least made the process seem less
daunting.
Published at DZone with permission of Nathan Smith, author and DZone MVB. (source)
