JavaScript promises

This is just a basic introduction to javascript promises. This is inspired by this article, which explains the intricacies very well.

javascript callbacks

One of the unique feature of javascript is the use of callback functions. Its extremely essential considering the fact that javascript is a single threaded language.

For those that have no idea what I am talking about, here is how callback functions are usually used.

function getData(callback){
    callback(loadDataFromSomewhere());
}

getData(processData);

When the getData is done, the result is passed into processData.

Callbacks are all well and good, until you start working on some complicated API project and all hell break loose. If you have been working with callbacks for a while now, you would probably have heard the term “pyramid of doom”, where your function calls stretches all the way to the right.

But there are more to [promises] than nice looking code. Using callbacks deprives us of basic programming features such as return and throw. More importantly, when something is wrong, there is no way to track down where that mistake is. All these results in bad code design.

Javascript promises

Promise, as defined by the A+ spec, is like an upgraded version of callback functions.

For most modern browsers, promises are implemented as window.Promise, but for those who are looking good polyfill, is one option you can look at.

Basic functions (then & catch)

If you use promises, there is two things you cannot do without, the functions then and catch. This knowledge should be sufficient to use most promise-based library (eg. pouchdb).

The function then accepts 2 parameters, the first is the function that the data is passed into if everything goes well, the second is triggered when some error has occured.

As for catch, it’s just a shortcut to write then(null, function). So when an error is thrown, the error object is passed into the function.

The promise stack

You can use promises as callback functions, but there is a much nicer way to format your code.

remotedb.allDocs(...).then(function (resultOfAllDocs) {
    return localdb.put(...);
}).then(function (resultOfPut) {
    return localdb.get(...);
}).then(function (resultOfGet) {
    return localdb.put(...);
}).catch(function (err) {
    console.log(err);
});

There is one special feature about promises. If you return a promise in the function, the result is passed into the next then function. This way, you can chain different sections of code together. (The allDocs, then the put, then the get).

The catch function at the end is a good practice. Putting it at the end ensures that whenever an error is thrown, it will be caught. Regardless of which level the error occurs, the error will still be logged.

Advanced functions (resolve & reject)

When you create your own library using promises, you tend to face the need to wrap your synchronous code as asynchronous code. Here is where resolve comes in.

function someAPI(){
    return Promise.resolve().then(function(){
        // some error might be thrown here
        return 'someSynchronousValue';    
    }).then(/* ... */);
}

This way, all your code will be promise-y, even the simple functions. This ensures code design consistency.

Besides, when you use promises this way, you can catch error easily. Gone are the days of slow and inefficient debugging.

So what about reject? The reject function returns a promise that is rejected immediately.

new Promise().resolve('some value').then(function(val){
    // do something to val
});

new Promise().reject(new Error('some error')).then(function(val){
   // do something to val
   // but this function will be skipped
}).catch(function(e){
    // this function is called immediately    
});

Promise.all (The foreach in promises)

There is another use case for promises. Let’s say we have a database query that returns a few rows, and we wanna update those rows. Typically, this is what the code will look like (in the minds of those non-promise ninjas).

getAllRows().then(function(result){
    result.rows.forEach(function(row){
        // update value of row
        row.val += 1;
        updateRow(row);              
    });
}).then(function () {
    // Naively believe all docs have been updated now!
});

But here is the right way to do it, with Promise.all of course.

getAllRows().then(function(result){
    return Promise.all(result.rows.map(function(row){
        //update value of row
        row.val += 1;
        return updateRow(row);
    }));
}).then(function(arrayOfUpdateRowResults){
    // all docs have been updated    
});

End

Well, all these are just the basics of using promises, there are certainly alot more to learn. If you are interested, I implore you to give this article a read. You will learn much much more.

Advertisements

Getting Started Series

The Getting Started series introduces you to fontend development. Follow these tutorials to learn the basics of web developement. With all these basics, you can make simple adjustments to your sites, add in new information on your own.

  1. Markdown – Simple formatting for documents
  2. HTML – markup language for the web
  3. HTML – forms and tables

But if you want a more complete tutorial, on how to design and implement a website design from scratch, contact ongspxm@gmail.com for more information.

pouchdb – mysterious “document update conflict” error

pouchdb is a fairly useful offline database library if you are developing web app that needs data storage. After looking through the api, I started work, playing around with it, thinking that it should be fairly easy to.

I tried to implement the mother of all data storage test – incrementing a stored value. This is first version of the code I have.

var db = new PouchDB('test');

db.get('1').catch(function(e){
        console.log(e);
    }).then(function(d){
        console.log(d.value);

        // if _id exists, pouchdb overwrites
        db.put({_id:'1', value:d.value+1});
    });

Well it seems like everything should work fine, especially when this is one of the most basic database function of all. But pouchdb is not simple technology.

I spent at least ten minutes struggling with this error here: document update error

What can be the problem? Everything seems to work fine. Google was my first go-to. I found this article by pouchdb’s author. I had a brief read, and it looks promising (promises, get it?). I plough through all 20 plus paragraph of the article.

JavaScript promises (not my solution)

After having a better understanding of how JavaScript promises work and how to use them properly, I tried to fix my code (the trial and error method).

My catch function is in the wrong place. Initially, I placed it there to catch the error where the entry is not found. But db.put may return an error too. The function should be placed behind in order to catch both errors, from both db.get and db.put.

After the changes, the second version of the code will look like this.

var db = new PouchDB('test');

db.get('1').then(function(d){
        console.log(d.value);

        // if _id exists, pouchdb overwrites
        return db.put({_id:'1', value:d.value+1});
    }).catch(function(e){
        console.log(e);
    });

Returning the db.put promise is crucial. It ensures that the catch callback is applied to it too. So if anything goes wrong with updating the file, we will know too.

I prayed hard and tried again. Still doesn’t work.

Keep updating it until it works (not my solution)

I scouted through Google again. This time, I saw the upsert plugin. As it turns out pouchdb is pretty raw and basic. Many of the high level functions have to be implemented by the developers themselves.

This function I have is something like a upsert function. You update when there is a record, you return error when it’s missing. And the plugin’s way of doing it is to keep calling the db.put function over and over, until the query finally gets completed.

Here is what I have after implementing the upsert function.

var db = new PouchDB('test');

function update(){
    db.get('1').then(function(d){
        return db.put({_id:'1', value:d.value+1});
    }).catch(function(e){
        console.log(e);
        update();
    });
}

Hopefully that will solve the problem. But no, I ran the script and I keep getting the update error, over and over again. Oh noes, let double check that code again. Why doesn’t it work yet, argh…

That small little detail right there

I really need some in depth search for this. When to pouchdb’s github repository and looked through the issues opened.

And to my pleasant surprise, I found this piece of gem. It looks exactly like the problem I am having. But what’s with all those _rev he is talking about? I delved into the JavaScript console and played around with the library.

Indeed, with each record, there is both a _id and _rev. After some intense googling and source code reading, this is what I have figured out.

Pouchdb uses the variable to keep track of its documents, so when updating the database, it must be included to let pouchdb know that it is the same record.

As such, it will be easier to alter the object returned from db.get and use it in db.put since all the required elements are there, intact, untouched. The code will look like this after the changes are made.

var db = new PouchDB('test');

function update(){
    db.get('1').then(function(d){
        d.value += 1;
        return db.put(d);
    }).catch(function(e){
        console.log(e);
        update();
    });
}

Why it works?

In order for pouchdb to sync with remote server, a git like system is implemented to help the database keep track of changes. So for each record there will be a _rev to keep track of the status of the document.

How does that help with syncing? When the document used to update the database has a different _rev value, it can mean two things — the database is seriously out of sync, or a document is being overwritten, both contributing to loss of data.

To prevent any thing from going wrong, pouchdb checks every record to make sure they are of the same version, hence the update error. To trick pouchdb into thinking that everything is alright, the object must contain the _rev variable, and the shortcut is just to make the changes to the record itself, which was in my final implementation of the function.

Tutorial – Creating a webchat with web.py

BitBucket Repository

I have previously written a tutorial on how to do long polling on webpy framework. Well, this is an updated version with a code repository that you can use to include an webchat subapp in your webpy applications.

Tools required

Frontend implementation – Layouts

For the main page, it would consist of one main area where the messages will be displayed. And an area at the bottom which consist of a type input, where the messages are submitted.

All the javascript coding will be included in the webchat.js, which will be described in more detail later.

/static/layout/index.html:

$def with (msgs)
<script src="static/js/jquery-1.11.1.min.js"></script><script src="static/js/webchat.js"></script>
<div id="msgs">
<h1>Messages</h1>
$for msg in msgs:
<div class="msg">$msg</div>
</div>
<div id="entry"><input id="message" type="text" placeholder="Message here ..." />
<input type="submit" value="submit" /></div>

The styling is not so important for this project, but you can view the CSS style in the code repository.

Storing the messages

We will be using sqlite to store our messages. So, open up your console, and type in sqlite3 database.db, and create the table using the following schema:

CREATE TABLE msgs(
msg_id INTEGER PRIMARY KEY,
msg_content TEXT NOT NULL
);

Simulating real-time events – Client side

To simplify our lives, we will be using jquery for the ajax calls. Everytime a call is returned, the message wrapper(#msgs) will be updated and another call is made.

Another function is needed to send the message to the server for storing into the database.

These functions will be included inside /static/js/webchat.js:

function sendMsg(){
var div = $('#message')[0];
var msg = div.value;

if(msg==''){
alert('Message cannot be blank');
}else{
div.value = '';
$.ajax({url:'/send?msg='+msg});
}
}

function longPoll(idx){
$.get(
url='/get?idx='+idx
).done(
function(data){
data = eval('('+data+')');
var msgs = data['msgs'];
if(msgs){
for(var i=0; i<msgs.length; i++){
$('#msgs')[0].innerHTML += "
<div class="msg">"+msgs[i]+"</div>
"
}
}
longPoll(data['idx']);
}
);
}

Simulating real-time events – Server side

Framework for the webpy application

This is the basic framework for webchat.py:

import web, time, json

urls = [
'/', 'Index',
'/send', 'SendMsg',
'/get', 'GetMsg'
]
render = web.template.render('layouts')
app = web.application(urls, globals())
db = web.database(dbn='sqlite', db='database.db')

class Index:
def GET(self):
return

class SendMsg:
def GET(self):
return

class GetMsg:
def GET(self):
return

if __name__ == '__main__':
app.run()

Serving the existing messages

Alter the Index class to read messages from the database and render it inside the layout:

class Index:
def GET(self):
res = db.select('msgs')
msgs = [r for r in res]

content = [m['msg_content'] for m in msgs]
return render.index(content, msgs[-1]['msg_id'])

Add new messages to the database

Insert the content using /send into the database:

class SendMsg:
def GET(self):
i = web.input()
if i.get('msg'):
db.insert('msgs', msg_content=i.get('msg'))
return
else:
return web.notfound()

Getting new messages – Long Polling

This is the important bit. The server side code for the long polling process:

class GetMsg:
def GET(self):
i = web.input().get('idx')
print i
if not i or not i.isdigit(): i = '0'

max_iter = 20; iter = 0
msgs = []
while not len(msgs) and iter'+i)
msgs = [r for r in res]
iter += 1
time.sleep(i)

if len(msgs): i=msgs[-1]['msg_id']

return json.dumps({
'msgs':[m['msg_content'] for m in msgs],
'idx':i
})

A bit of explanation here. The while loop and the time.sleep is what makes the whole idea works. The server will keep the request open until it detects a change in the database.

Running the server – Gevent WSGIServer

If you are running this as a subapp. Here is all you have to know. If you are running it as a standalone application, do read on.

Webpy’s default web server is not that great for production. Instead, we will be using the Gevent WSGIServer to run the application.

from gevent import monkey, pywsgi;
monkey.patch_all();

''' Whatever code you have '''

if __name__ == '__main__':
print 'WSGISever on 8080'
application = app.wsgifunc()
pywsgi.WSGIServer(('', 8080), application).serve_forever()

Fixing the static directory problem

The WSGIServer does not automatically serve static file like webpy. Fortunately, there is a fix for this:

urls = [
'/static/(.*)', 'Static',
'/', 'Index',
'/send', 'SendMsg',
'/get', 'GetMsg'
]

class Static:
def GET(self, file):
try:
f = open('static/'+file, 'rb')
return f.read()
except:
return web.notfound()

Going on from here

If you are lost in any of the steps above, do visit the code repository. It might be helpful to have a working sample where you can follow. Any troubles, drop me an email or leave it in the comments, I will help to be the best of my abilities.

BitBucket Repository

Web.py Application – User Authentication system

EDIT: Visit the tutorial here

Github Repository

After working on so many web designs, I have decided to hone my server side development skills for a change.

One of the most common web application system is an user authentication system. Many applications, regardless of its purpose, requires a user authentication system.

Using webpy, I developed a simple user authentication system, together with a few error handling pages and functions, which hopefully, can be altered to fit my future projects.

This project, though simple, took me more time than I had planned. I wasted quite some time looking into how webpy sessions work. There are a few troublesome portions with using sessions with webpy(tutorial here). But still I managed to get everything working fine, even completing a simple design for the interface.

Bitbucket Repository

Design Mockup – InsuranceSuite from 99designs

This design that I have chosen to implement is a content-based design. This differs significantly from other designs I have implemented, which focused more on graphics and aesthetics.

For the structure, I used a new css3 method for the layout. Flexbox layout greatly simplifies the structure of the layout design, reducing dependencies on pure css organization for the content layout. This could be very helpful, especially in creating mobile web applications.

That said, this flexbox structure certainly helps in content based designs, but for more graphical based design, it wouldn’t really do much difference. Therefore, future usage would have to depend on types of designs that I choose to implement.

The content of the mockup is quite scattered. I tried to implement as many design elements into the mockup as possible. Hence, I had to resort to using hidden links to activate different views (sidebar links toggles the form overlay), mashing up everything to give a complete feel of the design.

Original design:
Insurance Suite by JonSerenity

Design Mockup – SoundStation from 99designs

This latest design I have chosen is a mobile app design. This will be the first of many responsive design that I will be attempting. This is a very fun challenge, having to deal with all the changes in the breakpoint, coming up with the structure to display the different types of layouts.

Working on responsive interfaces is unlike working on a typical web design. With a web design, most of the time is spent on trying to replicate the exact look and feel of the design. But with a responsive design, a huge amount of thought is spent on figuring out the structure of the layout and the scripting required to change or update the layout accordingly. This could be a better practice for me, so you could expect more of these in the future.

Original Design (Click for mockup):
SoundDesign by Nataliia