How Microsoft Azure turns out to be really expensive because of hidden pricing

Few days back, a friend was complaining about rising bills from Microsoft Azure VM. They were simply trying to do load testing on one of their Azure VMs and bandwidth usage was causing inflated bills. I was flummoxed, and doubtful on Microsoft's ways of doing business, I decided to investigate the matter.

We use Digital Ocean as our VM provider, and we are extremely happy with their service. Most important of all, apart from being really cheap, they are very transparent with their pricing. So I compared a similar VM offered by both, Digital Ocean and MS Azure.

	Azure	Digital Ocean
Processor speed	2.1 GHz	2 GHz
No. of cores	2	2
RAM	3.5 GB	4 GB
Harddisk	60 GB	60 GB SSD disk
Price	$80 (Rs 5185.8)	$40 (Rs 2650)

Digital Ocean pricing details here: https://www.digitalocean.com/pricing/

MS Azure pricing details here: https://azure.microsoft.com/en-in/pricing/calculator/?scenario=full

If you notice, Digital Ocean provides a slightly better VM at exactly half the cost to MS. In fact, if you compare Digital Ocean with GoDaddy and AWS, the same conclusion can be obtained. Not a big deal, until you factor in the bandwidth costs of renting an Azure VM.

First of all, Azure calculators and pricing pages don't make it clear how much they charge for the bandwidth. On the other hand, Digital Ocean bandwidth prices are clear on their pricing page. I had to dig in a bit to find out the Bandwidth pricing of MS Azure and found it here:
https://azure.microsoft.com/en-in/pricing/details/data-transfers/

What I found was totally shocking. Analyze this:
MS Azure gives you 5 GB per month of bandwidth free, which is insanely low. What this means is that when you do anything non-trivial on MS Azure servers, you will easily cross the free limits and start paying for bandwidth. But more crazy is to find out how much they charge for bandwidth. They charge Rs 8.3 per GB over the 5 GB free limit (for South-East Asia). So if you end up using 4 TB in a month, you will have to shell out a huge amount of Rs 34,000 for that month.

Compare that to Digital Ocean, you get 4 TB of bandwidth free by default. Extra bandwidth is 2 cents per GB (Rs 1.3/GB). So Digital Ocean wins hands down in terms of cost effectiveness and transparency.

An important point is that if you are on MS Azure, and you happen to either get an app which becomes insanely popular or god forbid, you get hit by a Distributed Denial of Service (DDoS) attack, you will suddenly see a huge bill. The previous scenarios are a bit rare, a more common one is that as your app slowly grows in popularity, you may start to consume more and more bandwidth thus increasing your costs of the VM.

Hence I suggest that you look at Digital Ocean very seriously for your VM needs. Digital Ocean is a startup and has attractive pricing to lure customers. An important question then comes up: will Digital Ocean's price remain so aggressive at a later period as well? Even though nobody can deny that prices can go up in future, I still believe that Digital Ocean will always price their VMs lower to Azure, AWS etc. Their service is also very good.

One important caveat is that Digital Ocean provides only dedicated VMs, they don't provide other services like hosting of websites, databases, storage etc. Really speaking, for certain things like website-hosting, if you were to take a VM from Digital Ocean and host your website on it, it will be cheaper and better. Only problem is that you are on your own in terms of configuring your VM and setting up your website. Essentially, Digital Ocean keeps their pricing simple and aggressive by keeping their product really simple.

Another aspect is that once people get used to a particular VM provider, they rarely switch to a better one. This is true because people perceive switching as a painful task, though in actuality, in my experience, it was never very hard to switch. So, for this reason, you should be careful what VM provider you choose in the beginning itself. Going for a free VM under Microsoft Azure may appear to be a smart move, but as I stated above, you can easily be taken for a ride by being forced to pay heavily for bandwidth consumption.

General advice to young students about Startups

(This article has been derived from a talk I gave to Final year Computer Science students)

Before I start, I would like to ask you to think about the dream companies that you aspire to work for after graduation. You will probably come up with names of big companies. Have you ever thought how these companies were started? Have you thought about who were these crazy people who started these companies? If these people would have thought - let me join a big company and have a stable life, then these companies wouldn't have existed, right? Don't you think these people are special, not only because they create employment for others, but they also create a lot of value and wealth, not just for themselves but for countless others? The point of this thinking exercise is that instead of going to work for a big company, you can think of starting your own.

I just made startups look a bit glamorous, but let me tell you some important things. Startups are very challenging. Its like taking a roller coaster ride, there are lots of ups and downs. Sometimes, it happens, that while you are on the roller-coaster, you feel like puking. But when you are done with it, you feel like - hey, I want to do that again. Startups are also strangely addictive in the same way.

Another important thing - startups are really really hard. Let me give you a statistic - 90% of all tech companies fail. They close down. If that doesn't make your head spin, let me give you another one. 75% of all Venture Capitalist funded companies fail. If you know about Venture Capitalists, they only invest in companies that are already quite successful, that are already moderate sized. They mostly put in money for the company to scale. Imagine, these guys having all the money in the bank and still 3 out of 4 such companies fail. You might think once you have money, you will be successful. But surprisingly, that's not what always happens.

So you guys must be thinking - how is working in a big company different from working in a startup.

In a startup, there is lot of freedom to do things and a lot of learning happens. Lot of growth happens, because you will be doing multiple things at the same time. If you think in the short term and want financial stability, then go ahead, you can go for a big company. But if you think in the long term, if you think about learning and growth, then startups become a really good alternative.

One major complaint I hear about big companies is that there is lot of politics that happens. You need to suck up to the right people, or show that you are productive in some way, and that's how you get a promotion or a raise. Or perhaps somebody else does that better and gets ahead of you, which sucks, right? In startups, politics doesn't work. You have to just create something that your users want. You have to create a product that users love. You have to work hard, and that's what matters. Any tricks you do, simply stop to matter. I am thrilled that there is something is this world where hard work really pays. Startups in this way, are a great meritocracy.

Another major thing I hear about people working in big companies is that they are terribly bored of their work. Its not surprising to me. It happens that you get stuck in a role in a big company and you don't get interesting stuff to work on. In a startup, I have worked for my company for around 5 1/2 years, and I have to tell you that I have experienced all kinds of emotions like fear, anger, frustration, happiness, joy, thrill but I never experienced boredom.

In fact, if I were to define an entrepreneur in a simple sentence, it is someone who makes things happen. An entrepreneur is someone who keeps on doing things. How can you be bored when you keep on doing something or the other? So an entrepreneur does something, if that clicks, he will try to improve that. If it doesn't work, he will try to do something else. And that's how it works, an entrepreneur keeps on doing something and keeps on learning lessons along the way. As he learns, he becomes more mature and increases his chances of success.

So you might say - ok, I am interested in a startup. What can I do?

There are 2 most important people in a startup. The first is the people who do marketing and sales. In startup lingo - they are called the "hustlers". These people talk to customers, they try to sell the product. They also talk to customers and get their feedback as to how to improve the product. They hustle, and somehow get the product in the hands of the customer, they somehow get the customers to pay for the product. If you think that you are the kind of person who likes to talk to people, then you can become a hustler in a startup.

The other kind of people who are the most important in a startup are the technical people. They are also called "hackers". People like me. These are the people who write code. They make the web-app, they make the mobile app, they take care of the technology. They create the actual product that the customers use. I have to tell you that over the years, I have worked on different products, some failed and some were successful. At the end of it, it is highly satisfying to have worked on some products which people actually use. As computer science students, you should seriously look at this kind of role.

There is also another category of people - who are called the "hipsters". These people are the design people, they work on UX, UI design. They work on the website layout and the graphic design stuff. They are not as important as the other 2 people. But if you are into design, you can take this role.

Many a times, I goto startup events, I meet people and they say - hey, I am a marketing person and I am looking for a tech co-founder. Its never the other way round. So after long, I have realized that there is a huge demand for good technical people in Bangalore. And you may be surprised to hear that. Because you might think that Bangalore is the place where you have all the tech companies. You may think that this is where all the tech stuff is happening. But still, there is need for more technical people who are really good.

Here, I would like to tell you about a close friend of mine who works for a company called Accenture. Now, Accenture is a big company, an MNC, its a great brand to work for and you will think that working for Accenture is great. But the fact is - that he totally hates his job. And he has a technical profile. Can you guess why he hates his job? He is into Automated testing. He writes testing code for some shit code that somebody else has written. That's like the most boring job you can find. That's what happened, all the boring jobs have been exported from US to Bangalore. That's why, when you talk to IT crowd in Bangalore, they will tell you that Tech jobs are boring. Its because they are doing uninteresting and boring stuff like testing or BPO jobs. Here, I would like to give you some advice, that in case a company comes along and hires you for a testing job, try to convince them to give you a development role. If they don't agree and say that they are still going to make you do testing, try to negotiate with them saying that you will agree to do testing for 1-2 years, but after that, you will expect to work in a development role. Because working for years in testing is straight path to boredom.

If you want to become a good technology person, you may ask - what are the current hot technologies? What is that I should know to become good in the Technical domain. Nowadays, people are building web-apps. Like facebook, gmail are webapps. They are not websites, but they are complete applications running in the browser. So learning to write webapps would be great. For that you need to know HTML, CSS and most importantly Javascript. Because Javascript is the language that runs within the browser, so being good at Javascript will be a good investment of your time.

On the server-side for the webapps, different languages are used. We use Java, but you can use Java, PHP, Python, Ruby and others. Picking up any one of them will be good. Another thing, there are a lot of people carrying Android phones in India and lot of companies are investing in mobile app development. So, if you are good at Android development or iPhone development, then that's something really hot nowadays and you will not have any difficulty in finding opportunities.

Another question you might have is - when should I start my company? Is it immediately after college? Or is it a bit later.
There is no correct answer to this question. In the media, you must have heard of people like Mark Zuckerberg who dropped out of Harvard to work on Facebook and made it really big. The Snapchat CEO, Evan Spiegel, is the youngest billionaire at 24. So you might think that starting as soon as you graduate is the right thing to do. But in my opinion, starting a company right after college is really hard, unless you have some good mentors. So, working for a couple of years in another company is quite good. Or better, after you graduate, you should look for a job in a startup. This may jump start your career and may give you valuable experience, so that you can start on your own, at a later time.

Typical successful founders are 30+ years. So when you hear about successful people in their early twenties, they are the outliers, they are very few in numbers. Its later in life, that people become successful.

One important person I want to tell you about is Paul Graham. You probably wouldn't have heard of him, but he is very popular amongst the startup community. If you are even remotely interested in startups and entrepreneurship, you should check out his articles. They are all posted on his website, for free, and its a great resource.

Finally, I would like to conclude, that you guys being students, should have a lot of hunger for learning. There is a lot of exciting stuff that is happening within technology and if you dive into the right things, you will have a successful and satisfying career.

How to use Self-signed certificate in your Android app to communicate with the server

I have written a preliminary article to describe how SSL, Man-in-the-middle attack works plus what are the various options for you in case you want to use a self-signed certificate on your android app. In case you missed it, here's the link.

We have ODK-Collect that uses HTTPS connection to connect to a server and we were porting ODK to our OpenXdata server. ODK essentially doesn't allow HTTP connection to a server (unless they are on ports 443 or 8443). We also wanted to use SSL for security and for testing purposes we were using a self-signed certificate on the server. So lets begin with steps on how to get it to work on ODK-Collect (Please note that the same steps need to be followed in any other android app)

Step 1: Create your self-signed certificate and create .bks file

Use keytool to generate your key. It is in your Java bin path. In case, you already generated your keystore file, you can skip the first command.

keytool -genkey -alias handsrel -keystore ~/handsrelssl.keystore -validity 365

It creates a key with alias as handsrel, name of file as handsrelssl.keystore. It will ask for various things like passwords for the key and the keystore, the SSL details. Please note that Common name will be your hostname, in our case it was: handsrel.com

keytool -export -alias handsrel -keystore ~/handsrelssl.keystore -file ~/handsrelsslcert.cer

This command exports the key from .keystore file to .cer file.

keytool -import -alias handsrel -file ~/handsrelsslcert.cer -keystore ~/handsrelssl.bks -storetype BKS -providerClass org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath ~/Downloads/bcprov-jdk15on-146.jar

Here, you need to download the Bouncy Castle Provider jar file and give the correct location of where the jar is located.
NOTE: I had issues with the latest BouncyCastle jar file, since it produced an error. Reverting back to previous version i.e. 146 worked for me. You may need to do the same.

For windows users, replace ~/ with something like C:\somedir\mykeystore.keystore etc.

Success! We have got the .bks file which we will put in our Android app, which will allow our app to communicate with our server which has the Self-signed SSL certificate.

Step 2: Code changes

First, we need to put our .bks file in /androidappdir/res/raw/

We will write a new class that we will call as MyHttpClient which will extend DefaultHttpClient. This class will load our own trust store to check the SSL certificates, rather than android default trust store. Only when the certificate here matches on the server, will it work correctly. Here's how it looks like:

package org.odk.collect.android.utilities;

import java.io.InputStream;
import java.security.KeyStore;

import org.odk.collect.android.R;
import org.odk.collect.android.activities.MainMenuActivity;
import org.opendatakit.httpclientandroidlib.conn.ClientConnectionManager;
import org.opendatakit.httpclientandroidlib.conn.scheme.PlainSocketFactory;
import org.opendatakit.httpclientandroidlib.conn.scheme.Scheme;
import org.opendatakit.httpclientandroidlib.conn.scheme.SchemeRegistry;
import org.opendatakit.httpclientandroidlib.conn.ssl.SSLSocketFactory;
import org.opendatakit.httpclientandroidlib.impl.client.DefaultHttpClient;
import org.opendatakit.httpclientandroidlib.impl.conn.SingleClientConnManager;
import org.opendatakit.httpclientandroidlib.params.HttpParams;

import android.content.Context;

public class MyHttpClient extends DefaultHttpClient {
  
 private static Context context;
 
 public static void setContext(Context context) {
  MyHttpClient.context = context;
 }

 public MyHttpClient(HttpParams params) {
  super(params);
 }

 public MyHttpClient(ClientConnectionManager httpConnectionManager, HttpParams params) {
  super(httpConnectionManager, params);
 }

 @Override
    protected ClientConnectionManager createClientConnectionManager() {
        SchemeRegistry registry = new SchemeRegistry();
        registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
        // Register for port 443 our SSLSocketFactory with our keystore
        // to the ConnectionManager
        registry.register(new Scheme("https", newSslSocketFactory(), 443));
        return new SingleClientConnManager(getParams(), registry);
    }
 
    private SSLSocketFactory newSslSocketFactory() {
        try {
            // Get an instance of the Bouncy Castle KeyStore format
            KeyStore trusted = KeyStore.getInstance("BKS");
            // Get the raw resource, which contains the keystore with
            // your trusted certificates (root and any intermediate certs)
            InputStream in = MyHttpClient.context.getResources().openRawResource(R.raw.handsrelssl); //name of your keystore file here
            try {
                // Initialize the keystore with the provided trusted certificates
                // Provide the password of the keystore
                trusted.load(in, "YourKeystorePassword".toCharArray());
            } finally {
                in.close();
            }
            // Pass the keystore to the SSLSocketFactory. The factory is responsible
            // for the verification of the server certificate.
            SSLSocketFactory sf = new SSLSocketFactory(trusted);
            // Hostname verification from certificate
            // http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d4e506
            sf.setHostnameVerifier(SSLSocketFactory.STRICT_HOSTNAME_VERIFIER); // This can be changed to less stricter verifiers, according to need
            return sf;
        } catch (Exception e) {
            throw new AssertionError(e);
        }
    }
}

How MITM attack works and preventing NSA from snooping on your traffic

Nowadays, security has become a really big concern. If you are storing any user data on your servers, you should seriously consider encrypting all communications between your client and server using SSL.

Again, simply using SSL is not enough. There are lots of things people need to understand about SSL before they can assume that their SSL encrypted traffic is safe. One major thing that people need to watch out for is the Man-in-the-middle (MITM) attack. Let me explain how an MITM can be done.

Server ======> Client
               SSL

When a client interacts with the server through SSL encryption, all data is exchanged in a form which is not readable by anyone. Say, if a cracker was to listen to an already established SSL session, he will see encrypted data which he can't decrypt. In this case, the communication is fully secure.

But what if the cracker somehow inserts himself in the middle of the client and server even before the SSL connection was established?

Server ======> Cracker ======> Client
              SSL                        SSL

This scenario will mean that the cracker acts as server to the 'Client' and a client to the 'Server'. Thus the cracker can read all data being exchanged without having to bother about breaking the SSL connection. Please note that there still are 2 SSL connections being established, and both these connections are fully secure. But this kind of vulnerability can be exploited if the cracker can hijack the connection before it is established.

In order to prevent this kind of MITM attacks, the SSL certificates are required to be signed by a Certification Authority (CA). Say, a user tries to access data on the encrypted channel. The Server sends a certificate signed by a trusted CA, to the Client, and the Client already having a list of trusted CAs can verify that this certificate is indeed genuine. If a cracker tries to insert himself in the middle, the cracker will have to submit a certificate to the Client, but he can't get it signed by a trusted CA, so the Client will be shown an exception.

Remember, when your browser suddenly shows a warning about an untrusted HTTPS connection, and asks whether you want to add an exception? In devices like Android, the SSL connection will not be established at all, since the certificate was not signed by a trusted CA.

There are various things that we can deduce out of this:
1. This prevents MITM attacks effectively, except in rare cases which I will get to in a short time.
2. If you have a self-signed certificate on your server, then there is no way the Client can know whether he is being a victim of MITM attack or not. (since the cracker in the middle will also have a self-signed certificate). Thus self-signed certificates in production mode are a strict no-no, unless you can send your self-signed certificate to the user in a separate secure channel.
3. If you have a self-signed certificate on the server, and you just want to use in testing mode in your app, your client app will not allow SSL connection to happen.

Particularly, we were faced with the issue that we were trying to run an android app called ODK-collect with a self-signed certificate on the server (for testing). Since android didn't see the certificate as trusted by a CA, it didn't allow the connection to take place. As explained before, this is by design to prevent MITM attacks.

If you are asking why I would like to have self-signed certificate on the server when I previously recommended against it, there can be various reasons for doing that:
1. Maybe you are stuck with talking to a server which uses a self-signed certificate. You have no control over the server but are working on an app that has to talk to this server.
2. You want to test your client-server connection using self-signed certificate. I am not sure if you can obtain a CA signed certificate for hostname as localhost.
3. Even free certificate provider like CAcert may sign your certificate, but CAcert is not treated as trusted CA by most browsers and clients, so they are treated at par with a self-signed certificate.
4. Maybe you just want to avoid the hassle of paying for a CA to sign your certificate, and want to go for self-signed approach.

So, we were trying to find out how to use self-signed certificate and getting ODK-collect (any android app for that matter) to communicate with our server. There are various options available:

1. Trust all certificates
Stack Overflow and other blogs are filled with answers that will allow your app to trust all certificates by default. Even though most answers say that this should only be done in testing mode, what if somebody forgot to turn it off in production mode? In case you are wondering, if your android app trusts all certificates, then your client will never be able to detect MITM attack and will continue to do business as usual. So, in case you want to do it like this, you should exercise extreme caution that this code doesn't go in production. In fact, if running your app with your own self-signed certificates is so easy, why even do this?

2. Install your certificate using SD card method
You can install the certificate using SD card method. If you are testing on a few devices, yes, you could do this. But if you are using this on many devices, there is the problem of having to do this manual method. In case you are going to use self signed certificate in production, you will have to ask all your users to manually install the certificate. Hardly a feasible scenario.

3. Packaging the certificate within your android app and allowing an SSL connection only with the same certificate, disallowing all others
This is the preferred route. What you do is - put the certificate within your apk, change your HTTPS connection to only allow this certificate and discard the rest.

Let me come back to an important issue that I didn't address previously. Say, you have a certificate signed by a trusted CA on the server, you may think that you are now fully secure. But unfortunately, there is a scenario when the cracker could get hold of a fake certificate signed by a trusted CA, and do an MITM. The end-points of the connection channel have no way to know that their traffic is being snooped by the cracker. You think this is highly unlikely? What if the cracker turns out to be an influential organization like the NSA?
Read this: https://www.schneier.com/blog/archives/2013/09/new_nsa_leak_sh.html

You may say that NSA is a spying agency that is trying to protect American interests. But what if there was some spurious government organization that uses its own influence to make some trusted CA sign their fake certificates? Definitely possible. Worst part is, there is not much you can do to prevent such kind of an attack. This is precisely why people like Schneier are saying that the Internet is broken.

But if your android app only accepts your self-signed certificate, and not even certificates from a trusted CA, then this kind of influential attack can be prevented, since you are only allowing your own certificate for communication, nothing else. Note that the certificate has to reach the user in a safe and secure manner, in this case, packaged within the android package (.apk). Another note: you have to ensure that when the user downloads your apk, it is not changed in transit by a cracker (use checksums etc. to verify the downloaded apk). We are currently using this methodology for our new product called Form Factor.

Enough been said. In my next blog post, I will talk about what code you need to write to get this done. This will not only apply to ODK-collect, but any android application where you want to get running with self-signed certificate on your server.

Review of DOMO Slate X3D

I was recently looking for a tablet and after looking at some reviews, I decided to buy DOMO Slate X3D. Still I was dissatisfied with the fact that there aren't any comprehensive reviews for this tablet. So I thought, I should take the plunge and help people out in making the decision whether this is the one to buy or not.

First impression - the tablet, in terms of build quality looks okayish, but obviously it can't matchup to the iPad and Samsung galaxy tab lookswise. The tablet is a bit thick, reminds me of the old samsung galaxy tab.

The biggest turn down for the tablet is its screen. It doesn't look very pretty, and the reason for that is the colors. Unlike other smart phones and tablets which come with 16 million colors, this has less colors, and it is not specified. But let me take a guess - 256k colors? The screen is also a bit too wide making movies appear a little skewed, and browser in landscape mode leaves less space for viewing, especially when you have the virtual keyboard coming up.

Nowadays most cheap and low end tablets come with resistive touch screens. I have used resistive touch and am convinced that resistive touch is really crappy. So what I say is - if you are tempted to save money by going for a resistive touch screen, please 'resist' that temptation. It is just not worth saving the extra money.  This tablet thankfully has capacitive touch, and it is actually good, almost unexpected after the initial disappointment. The virtual keyboard is really excellent to use, and no touch on any keys is lost - a good typing experience and excellent capacitive touch here.

I tested out the sound card and compared the sound with my other devices like laptop, smart phone and my take is that sound card is exceptionally good, with very good loudness and decent sound quality too. Mind you, I am talking about the output to my head phones, and not the built in speakers. The built in speakers, sadly, are not very loud and are a bit disappointing.

This tab has a front facing camera, appropriate for skype video chats or video conferencing. The camera is good. It takes photos with resolution 640x480 (It is .3 MP camera, a spec that they have not specified anywhere)
On the other hand, the mic is very disappointing. I downloaded some recording software and tried recording, and the mic doesn't register any sound unless you put it very close to your mouth. What this means is - the tablet can't properly be used for skype calls because of poor mic.

OK, this is a gaming tablet, so I got to talk about games as well. I am not much of a gamer, but I don't mind the occasional hurling birds at pigs or slicing some fruits. I also tried a car race on the tablet which came pre-installed. And my take on games is - this tablet really lives upto expectations. There is no lag, the extra graphics processor comes into play and the gaming experience is very nice.

I downloaded adobe pdf viewer on the tab, and opened some comics and was satisfied with the way it rendered them. So this is a good device for reading books and articles.

Battery is nice, gives a decent backup of 5-6 hours as specified, on continuous usage, enough for me who is going to use this tablet mainly at home, not far from a power outlet.

This tablet has the latest android OS, and it comes pre-rooted too. That may not mean much to a normal user, but to a developer that is added convenience. Even if you are not a developer, you are still in advantage because you have many benefits of rooting like ability to make complete backups, and many useful things that a rooted device can do which a non-rooted device can't. Not to say that it is difficult to root a device like Samsung tab, but still, you stand the risk of losing your warranty from the manufacturer if you root, which is not the case here, since the manufacturer himself rooted the device for the user.

I tested times for shutting down and starting up the tablet. It shuts down in a real quick 2-3 seconds, and takes just over a minute to do a complete start of the Android OS.

512 mb ram, 4 gb internal memory is enough to run any app and install as many apps as anybody wants without worrying about space limitations. Not to forget that you can easily add an extra sd card and increase storage space.

This device comes with only a charger, no cables to connect to pc or usb drive (meaning you have to buy them separately). But considering the cost of the device, I am not complaining.

This tablet may be one of the rare ones that is cheap as well as good. Apart from the screen which should have been 16 million colors, and a poor mic, everything is awesome and for a tablet of around 8k range, this is an excellent buy. But I would like to add that if you are not on a very tight budget, do look at Samsung tab 2, which has more features, better build quality, beautiful screen and everything good and prices are expected to goto to about 15-16k soon.

New localstorage based database API

We were initially working on Firefox 4.0 and Indexed DB, and I thought of writing some tips and tricks on getting it working. It turns out that Indexed DB is not yet supported on Android browsers and iPhone browser, which is a shame, because we want to run our web-app throughout the whole ecosystem of devices that are now available in the market.

Most browsers support a basic subset of HTML5 standards. But Indexed DB, surprisingly, is not yet heavily adopted. So we were faced with the dilemma of how should we get our app running on Linux, Windows, Mac, iOS, Android etc.

We looked at Web database and Indexed DB, both of which have serious limitations, as of now. So I decided that something had to be done. And I clicked upon a great idea of writing my own API for storage, that works on localstorage.

Now local storage is being supported by all browsers. But it is a very simplistic approach of storing client side data in key-value pairs. This means that you can probably store a To-do list using it, but you may not find it convenient for having something like an electronic data-collection app storing electronic forms, data filled in for the forms, users, settings all in local storage. All this complexity needs a database like storage and organization in form of object stores or tables.

So the local storage API came into being. What it does is - it is just a layer above the localstorage of the browser, so that a developer can write statements to create a table, add columns, add records, retrieve records, search on the records, drop tables, drop full database and more. The API actually stores all data in certain specific key formats so that organization is possible and the API can easily do the operations required by the developer.

The advantages of this approach are manifold, some of which were already mentioned, but here they are again:
1. Requires local storage to be working on the browser, something that's supported on all major browsers right now, including mobile phone browsers.
2. Local storage is synchronous in nature. I am not sure since this could be a disadvantage for some, since they may be looking at asynchronous operations. But the thing is, our app needed synchronous operation, and it much easier to write code with synchronous requests.
3. Is somewhat like Indexed DB in implementation, no sql knowledge is needed.
4. Fairly easy to use the API

For more details and for downloading it, please go to:
https://code.google.com/p/gwt-localstorage-db/

Indexed DB tips and tricks (For Firefox 4)

In these days, webapps are becoming more and more popular and we have a strong case for webapps to replace even desktop apps. For one, webapps run the same way on different operating systems. They just need a browser for running.


After coming up of HTML5 spec, we are looking at really powerful webapps which can have the ability to work offline. Different browsers try to implement HTML5 in their own manner, which is a problem for developers. Especially, since HTML5 is new, there are lot of uncertainties in this area.


One important thing I came across recently is the Indexed DB spec that Firefix 4 and other browsers are implementing. They have made a big change from the Gears implementation (Sqlite) as well as the WebSQL implementation (Sqlite again). Sometimes it is difficult to find proper documentation in this regard.


Since we are currently working on getting IndexedDB working on Firefox 4, we went through a lot of Mozilla documentation and found out that it is quite confusing for a person to understand IndexedDB API.


So I thought that I should share my experiences in this regard, and give some tips and tricks, and some sample code on how to get started.


1. First thing first, let me give you the correct links for the documentation:
https://developer.mozilla.org/en/IndexedDB/IndexedDB_primer

This is an excellent IndexedDB primer from the Mozilla team. It also gives links to other important documentation like the Mozilla IndexedDB API and W3 spec.

Other documentation found on sites like HTML5 rocks etc. sadly don't work on Mozilla Firefox4.
http://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/
This is also grossly outdated, and I wasted a lot of my precious time in trying to run the code given here.


2. Note that creating or opening a db on the Firefox 4 has a little different way of doing it. You write a call through mozIndexedDB, not window.indexedDB


var request = mozIndexedDB.open("MyTestDatabase");

To do error handling at one place, so that you don't have to do it repeatedly, you can include this code in the onsuccess() method for opened db.

db.onerror = function(event) {
  // Generic error handler for all errors targeted at this database's
  // requests!
  alert("Database error: " + event.target.errorCode);
};

3. When you try to run IndexedDB through java script embedded in a local HTML file, you will get nasty errors : errors of type unknown and it won't work whatever you do. But if you put in on a web server, then the same html file works and your IndexedDB will be happily created.


One way to create local IndexedDB so that you can test out your applications is: we created a GWT project, and put all our JavaScript code inside a script called from the HTML file inside the GWT project. Then we ran the GWT project and sure enough, it worked well. (I have to say here that I have been working on Java and GWT for quite some time, and am relatively new to JS and JSNI)


4. On Firefox4, if you want to debug your JavaScript code, you need to goto
Tools->Web Console
and this should bring up the web console where you get all the error messages.


5. We tried writing all IndexedDB code as a JSNI function inside our Java classes (since we are coding in GWT or Java), and we had lot of problems in certain JS commands like delete or const. So there is a very simple way to include your IndexedDB code as JS script and still access it from your Java code. Let me give you short steps on how to do that:


First, write all your JS script files inside the main HTML file or access it there using something like:

<script type="text/javascript" src="createDB.js">
</script>
where createDB.js contains the code for creating database and objectStores (say there is a function called createDB() inside the .js)


Now you can easily access this file from your HTML file, by just including the following code:

<script type="text/javascript">
createDB();
</script>

Ah, but this is not very impressive, since we want to access this function inside your Java class, not through your HTML file. So, in order to do that, you do something like this in your Java file:

public void onModuleLoad() {
createDb();
}


public native void createDb() /*-{
$wnd.createDB();
}-*/;

And voila, you have all functions defined inside .js file accessible through JSNI in your GWT java file.


You can pass objects from Java to JS, JS to Java etc. without any problems from now on.


6. An important thing to note is that you will have to open the database every time you want to do a transaction on the Indexed DB. It seems like that the db closes itself once you are done with the transaction and you have no more code for it in that particular function. This is a bit annoying since it increases the size of your code, but my guess is that this was deliberately done to prevent memory leakages or having unsaved changes on the db.


7. In case you were wondering how to specify an object store with autoIncrement as true, here is some sample code:

var user_form = db.createObjectStore("user_form", {keyPath: "user_form_id", autoIncrement: true} );

Then you can create indexes inside the createIndex command like this:

user_form.createIndex("name", "name", { unique: false });

unique:true will make the column as a unique column type, so that it is always unique.

8. Some people may tell you that the 'put' function is something like the 'add' function for adding the data to the Indexed DB. That is a misconception, since 'put' has some very important differences that should be carefully noted. 


What they tell you is that you can use 'put' function to either add or replace a record in the db. But what they don't tell you is that 'put' function needs the Primary key as an input for it to work.
Let me explain by example. If suppose, your object store(table) had an auto increment key as Sno and a data column called Name.
In order to use the add function, you can just specify Name to the function and it will auto-generate the key by itself. But if you use the 'put' function here, it won't work and it will show an error. And that goes for even if the Name you specified is already existing. For doing a put, you will have to specify the key. So, if you really want to auto generate the key, you have no option but to use the 'add' function, and if you want to edit certain location, then you have to do a search and find what is the key for that particular data, and then pass it on the 'put' function.
Yes, I know this is going to make your code longer, but I don't think there is any other method to doing this. If you have a situation when you don't know whether you have to do an 'add' or a 'put', then you have to first search for a particular index, if found, do a 'put' at the found key, else do an 'add'.

That's it for now. I will keep on updating with more tips and tricks as I learn more while getting the Indexed DB working on FF.