by Bozho | Jul 15, 2015 | Aggregated, Developer tips
Tomcat has a couple of connectors to choose from. I’ll leave aside the APR connector, and focus on the BIO and NIO. The BIO connector (blocking I/O) is blocking – it uses a thread pool where each thread receives a request, handles it, responds, and is returned to the pool. During blocking operations (e.g. reading from database or calling an external API) the thread is blocked. The NIO connector (non-blocking I/O) is a bit more complicated. It uses the java NIO library and multiplexes between requests. It has two thread pools – one holds the the poller threads, which handle all incoming requests and push these requests to be handled by worker threads, held in another pool. Both pool sizes are configurable. When to prefer NIO vs BIO depends on the use case. If you mostly have regular request-response usage, then it doesn’t matter, and even BIO might be a better choice (as seen in my previous benchmarks). If you have long-living connections, then NIO is the better choice, because it can server more concurrent users without the need to dedicate a blocked thread to each. The poller threads handle the sending of data back to the client, while the worker threads handle new requests. In other words, neither poller, nor worker threads are blocked and reserved by a single user. With the introduction of async processing servlet it became easier to have the latter scenario from the previous paragraph. And maybe that was one of the reasons to switch the default connector from BIO to NIO in Tomcat 8. It’s an important thing to have in mind, especially...
by Bozho | Jun 23, 2015 | Aggregated, Developer tips
A blue-green deployment is a way to have incremental updates to your production stack without downtime and without any complexity for properly handling rolling updates (including the rollback functionality) I don’t need to repeat this wonderful explanation or Martin Fowler’s original piece. But I’ll extend on them. A blue-green deployment is one where there is an “active” and a “spare” set of servers. The active running the current version, and the spare being ready to run any newly deployed version. The “active” and “spare” is slightly different than “blue” and “green”, because one set is always “blue” and one is always “green”, while the “active” and “spare” labels change. On AWS, for example, you can script the deployment by having two child stacks of your main stacks – active and spare (indicated by a stack label), each having one (or more) auto-scaling group for your application layer, and a script that does the following (applicable to non-AWS as well): push build to an accessible location (e.g. s3) set the spare auto-scaling group size to the desired value (the spare stays at 0 when not used) make it fetch the pushed build on startup wait for it to start run sanity tests switch DNS to point to an ELB in front of the spare ASG switch the labels to make the spare one active and vice versa set the previously active ASG size to 0 The application layer is stateless, so it’s easy to do hot-replaces like that. But (as Fowler indicated) the database is the most tricky component. If you have 2 databases, where the spare one is a...
by Bozho | Jun 6, 2015 | Aggregated, Developer tips
Sometimes your project may have optional dependencies. E.g. “if apache http client is on the classpath, use it; otherwise – fallback to HttpURLConnection”. Why would you do that? For various reasons – you may be distributing a library and you may not want to force a big dependency footprint. On the other hand, a more advanced library may have performance benefits, so whoever needs these, may include it. Or you may want to allow easily pluggable implementations of some functionality – e.g. json serialization. Your library doesn’t care whether it’s Jackson, gson or native android json serialization – so you may provide implementations using all of these, and pick the one whose dependency is found. One way to achieve this is to explicitly specify/pass the library to use. When the user of your library/framework instantiates its main class, they can pass a boolean useApacheClient=true, or an enum value JsonSerializer.JACKSON. That is not a bad option, as it forces the user to be aware of what dependency they are using. Another option, used by spring among others, is to dynamically check is the dependency is available on the classpath. E.g. private static final boolean apacheClientPresent = isApacheHttpClientPresent(); private static boolean isApacheHttpClientPresent() { try { Class.forName("org.apache.http.client.HttpClient"); logger.info("Apache HTTP detected, using it for HTTP communication.); return true; } catch (ClassNotFoundException ex) { logger.info("Apache HTTP client not found, using HttpURLConnection."); return false; } } and then, whenever you need to make HTTP requests (where ApacheHttpClient and HttpURLConnectionClient are your custom implementations of your own HttpClient interface): HttpClient client = null; if (apacheClientPresent) { client = new ApacheHttpClient(); } else { client = new...
by Bozho | May 14, 2015 | Aggregated, Developer tips
You may be familiar with the “noisy neighbour” problem with virtualization – someone else’s instances on the same physical machine “steals” CPU from your instance. I won’t be giving clues how to solve the issue (quick & dirty – by terminating your instance and letting it be spawned on another physical machine), and instead I’ll explain my observations “at scale”. I didn’t actually experience a typical “noisy neighbour” on AWS – i.e. one instance being significantly “clogged”. But as I noted in an earlier post about performance benchmarks, the overall AWS performance depends on many factors. The time of day is the obvious one – as I’m working from UTC+2, my early morning is practically the time when Europe has not yet woken up, and the US has already gone to sleep. So the load on AWS is expected to be lower. When I experiment with CloudFormation stacks in the morning and in the afternoon, the difference is quite noticeable (though I haven’t measured it) – “morning” stacks are up and running much faster than “afternoon” ones. It takes less time for instances, ELBs, and the whole stack to be created. But last week we observed something rather curious. Our regular load test had to be run on Thursday, but then and till the end of the week the performance was horrible – we couldn’t even get a healthy run – many, many requests were failing due to timeouts from internal ELBs. Ontop of that, spot instances (instances for which you bid a certain price and someone else can “steal” from you at any time) were rather hard to...
by Bozho | May 8, 2015 | Aggregated, Developer tips
Log collection is essential to properly analyzing issues in production. An interface to search and be notified about exceptions on all your servers is a must. Well, if you have one server, you can easily ssh to it and check the logs, of course, but for larger deployments, collecting logs centrally is way more preferable than logging to 10 machines in order to find “what happened”. There are many options to do that, roughly separated in two groups – 3rd party services and software to be installed by you. 3rd party (or “cloud-based” if you want) log collection services include Splunk, Loggly, Papertrail, Sumologic. They are very easy to setup and you pay for what you use. Basically, you send each message (e.g. via a custom logback appender) to a provider’s endpoint, and then use the dashboard to analyze the data. In many cases that would be the preferred way to go. In other cases, however, company policy may frown upon using 3rd party services to store company-specific data, or additional costs may be undesired. In these cases extra effort needs to be put into installing and managing an internal log collection software. They work in a similar way, but implementation details may differ (e.g. instead of sending messages with an appender to a target endpoint, the software, using some sort of an agent, collects local logs and aggregates them). Open-source options include Graylog, FluentD, Flume, Logstash. After a very quick research, I considered graylog to fit our needs best, so below is a description of the installation procedure on AWS (though the first part applies regardless of the...
Recent Comments