I noticed something while looking at our server logs; a large number of exceptions are thrown as a result of the googlebot visiting the pages where they occur.. So it is allowing us to find obscure bugs that are data-specific that will be a nightmare to find otherwise. While the impact is low, I thought it was amusing to see google doing some of our QA for us.
A couple of days ago, I gave Ajay really bad advice. He was changing a piece of code to order the results by created_at instead of ids because ids and my code review boldly stated:
Ids will be chronologically ordered 99.9999% of the time the ids will be chronologically ordered.
While this might be true right now, it is really suboptimal. I realized that when I was thinking about the best way to setup our new database server. It occurred to me that we might want to setup a master-master replication scheme for MySQL server. Once we do that, each MySQL master will have its own range of ids to use when assigning a new automatically generated id to a record being inserted. For example, server A will be able to use ids within the range 1 - 99999 and server B 100000 to 199999
Once that happens, the chronological order of ids will be less continuous and might look like: 324, 100231, 325, 326, 100232,et.
So the lesson I admit to learning is: never sort by IDs when you mean to sort by date. In fact, I can’t think of any reason why you would ever want to sort by ID.
Cache is money to MedHelp as it helps us scale by reducing cpu, I/O, roundtrip times, etc. This is critical as we continue to grow from millions of users to tens of millions of users and beyond.
There are five popular ways to cache:
Page Cache
Page caching caches static pages so the request does not need to hit your rails server. I can imagine, every page on wikipedia is page cached and invalidated on update. This has huge performance benefits as we can configure our web server in front of rails to return the cached html page. We could use page caching for our medical dictionary pages. Almost all our pages have or will soon have dynamic content so the value of page caching appears low.
However, an interesting use of page caching is to leverage the fact that all pages viewed by a non-logged-in users looks the same. Since, the majority of our traffic are new users from google this will have a large impact. The trick here is that we have to identify that a request is from a non-logged in user from inspecting the cookie in our front-end web server.
Methods: caches_page and expire_page or sweepers
Action Caching
In action caching, the request hits the rails server and passes through all the filters. This is useful when you require auth to differentiate logged in and out and have an authorization filter. I cannot think of a case where we would use this form of caching since we use dynamic content on almost every page.
Methods: caches_action, expire_action
Fragment Caching
Fragment caching saves rendering of portions of your view. This method is interesting as we could cache the middle div (user journals) in the people page and invalidate using a time to live.
Another interesting technique, is to use identifiers on the cache fragments so that in the controller you do not execute code related to that fragment (ie, save db calls and computations). This does poke a hole in the MVC model as it creates an additional coupling between view and controller. Here is a code example:
View:
<% cache( :controller => :post, :action => :show, :subject_id => @post.subject_id ) dp %>
<% # beautifully written medhelp code %>
<% end %>
Controller:
def show
unless read_fragment( :controller => :post, :action -> :show, :subject_id => @post.subject_id )
# lots of db calls
end
# code you need for other fragments on the page
end
def edit
...
expire_fragment(:controller => :post, :action -> :show, :subject_id => @post.subject_id)
end
Methods: <% cache do %> in the view expire_fragment
Model Caching
Model caching is when we store ActiveRecord and db results to save db calls which is typically the bottleneck of our web appplication. We currently use this everywhere and we are very strict in reviewing this in our bottoms up design reviews and code reviews prior to check-in. For example, we cache all users. Soon, we will be caching all posts, subjects, and forums. All hail memcache.
Methods: CACHE get/set, act_as_cachable
In-memory computational cache
This is where we save our computation in a data structure. I almost forgot this since it is second nature. An example of this is caching the key words -> links data structure that is used to link-ify user generated text. See our medical terms highlighting in user journals and forum posts.
Methods: CACHE get/set, session set/get, class variables