Threadpool and the task queue

Every architecture makes way for threadpool and a task queue.  Multiple thread wait on the queue always ready to pick up the next task and execute them. Once implemented, the next task is tuning it. How many threads? What is the size of the queue? Blocking queue or throws error on full? Retry handler?

Before you start worrying about this ask a simple question. How much time does it takes to execute the task? If it is not at least couple of orders of magnitude greater than time it takes to do context switch, don't bother about the threadpool/queue, just execute it right there, on your current thread.

Here is why?

• Task queue has a lock. More threads and more often it is accessed, more contention, more time to submit the task. Extra context switch just to acquire the lock. Basically you are doing serialization before getting to parallelism here. More threads + more tasks => more time per insert. Think of it like talking to a customer care executive(CCE). You do lots of IO using IVR and finally reach the CCE and the guy instead of answering your questions connects you to another guy and you need to explain the problem once again. That is pretty much how context switch works. If you need to talk for 10-20 minutes, it might be worth it, but if all it takes is few seconds of conversation, it just wastes time.   
• Once the task is submitted, it need to wakeup some thread. That is context switch, costs time.
• By the time this new thread wakes up because of lock and time elapsed most of the variable it needs are out of cache...more time. Read lock semantics for JVM. 
• How do you do error handling from the task? Extra code, extra states.

You can avoid all this by executing the task inline....normal function call. It will run faster.  It is easy to write/debug. The assumption here is that the task really takes short time to execute and it mostly cpu intensive. Webserver using threadpool is understandable. Single request might need to do file IO, access some locked resources, possibly make multiple database queries. These are kind of things that make sense in threadpool...things that are complex enough to be simplified by using  a new/dedicated "thread of execution".  For other things, function call is the most efficient.  

Out of select

Some non-blocking architectures use separate threads for IO and others as worker threads. One problem with this design is extra latency because even though the worker is done with its work, the IO thread is blocked on select/epoll. One simple way to wake up selector under such conditions is to open a pipe per selector thread. The read end is made part of selector fd set and the write end is used by the worker thread to write one byte on the pipe which wakes up the selector. This ensures that whenever new fd needs to be registered with select, it wakes up as soon as possible instead of timing out on read timeout.

Adsense for TV adds

TV adds is $64 billion industry in USA alone.  And it works on the TRP rating systems which are nothing but statistical formulas applied on TV viewing timings of few thousand homes per country.  The simplest way to disrupt this market is by controlling the remote control. A wifi-enabled Android/iOS running remote can give real time information about who is watching which channel at what point in time.  And once you have it, you already have Adsense for TV.

And that is what I think Apple and Google are going to fight for over the next few years.

What is the mystery “entertainment device” Google is testing?

Apple patents new touchscreen remote control for a future Apple TV

IntoNow, the ipad app purchased by yahoo was a nice step in this direction using the audio matching technology. I guess this whole set of social TV apps is mostly about finding who is watching what at any point in time.

The same objective can be fulfilled by making TV's more intelligent (Samsung TV apps) and also by network connected setup boxes.  But I feel universal remove is a much cleaner and simpler way to go about doing this. If google were to crack this, they will not only be the entry point to anything we do on the browser but also entry point to all devices we use in our houses.  When we switch on/off lights, how many times does microwave is used, how much TV we watch, etc.

It will be interesting to see how they market this and at what price points.
 

Cacheismo learnings

I already knew lua, memory management, writing servers and other technical bits. I did learned the automake stuff to make sure people can compile it.

But the best part was something else. Marketing. I guess I failed miserably at that one. I don't know if anyone has downloaded cacheismo code and tried to compile it or is anyone using it. I think it is one of my best works and it is free and I don't know what more do I need to do to convince people that it is better that memcached.

I tried the following:

• I wrote a mail on memcached group explaining cacheismo.
• I wrote to author of highscalability.com blog. He was kind enough to include a link. 
• I created cacheismo google group. Only my friends joined. (Thanks!). No questions so far.
• I tried to answer some of the questions on stackoverflow about memcached. I looked at problems which people face but can't solve with memcached. Tried answering the questions to best of my knowledge and also provided information about how it can solved using cacheismo. Someone removed all my posts :( from stackoverflow. 

So I guess even if their exist people who might find cacheismo useful, it is kind of impossible for them to find it, unless of-course they magically search for cacheismo on google.  So the question is what is the plan? And the answer is nothing much. I am not actively working on cacheismo. I will be more than happy to help anyone who wants to use it.  I need to solve the discovery problem and the plan there is to keep posting to stackoverflow...until the person who deletes my posts gets tired of it. Quora is another option. And may be some videos on youtube.  

May be caching is not such a big problem for people and memcached is good enough. Well in that case I will write some more servers. Http Proxy something like haproxy but configurable in lua might be fun. Or may be websocket server for HTML 5 applications.  

Loan or Subscription

Some things are expensive. If sold only at their price points, they would have a very small addressable market. Even if they are useful, even if they would provide move value over time than what they cost, making a choice to buy them is fairly hard.

The first solution to this problem was delayed payments. Companies will allow customer to settle bills later, helping them in maintaining cash flows. The second solution was loan from banks and the latest one is credit cards.  All these mechanisms allow customers to have something of value now and pay for it later (hopefully realizing the value of the thing they purchased). But the risk of not being able to realize the value stays as the purchase is not reversible.

In my opinion the best solution to this problem is service model, pay as you use. Many of the severs used in companies have vanished. The email server, the content management system, the svn repository, etc. We have gmail for email, google docs for content, github for svn and all of these are on services model. 

I guess subscription helps in figuring out the "price" of something. Figuring out lifetime value delivered by a product is hard. By forcing ourselves to think about value of something in a month, we have better chances of falling in range that is acceptable to the customers. The second benefit is frequency of choice. We give our customers every month a chance to stop using us and selecting someone else. This has two benefits. One it makes decision making simple for the customer and second it brings focus to the company because we need to keep "selling" ourselves, keep delighting the customer, to be in business.

Software is not the only thing that can be sold as a service. Almost everything can fit in this scheme of things. If you are selling something expensive that provides its value over time, subscription is your best answer to "barrier to buy".  Product/Service is a superficial packaging over the value you provide. Package your value the way it makes sense for your customers.