by Bozho | May 3, 2015 | Aggregated, Opinions
I just watched Citizen Four, and that made me think again about mass surveillance. And it’s complicated. I would like to leave aside the US foreign policy (where I agree with Chomsky’s criticism), and whether “terrorist attacks” would have been an issue if the US government didn’t do all the bullshit it does across the world. Let’s assume there is always someone out there trying, for no rational reason, to blow up a bus or a train. In the US, Europe, or anywhere. And with the internet, it becomes easier for that person to find both motivation and means to do so. From that point of view it seems entirely justified to look for those people day and night in attempt to prevent them from killing innocent people. Whether the effort to prevent a thousand deaths by terrorists is comparable to the efforts to prevent the deaths of millions due to car crashes, obesity-related diseases, malpractice, police brutality, poor living conditions and more, is a non-technical question that I’m not qualified to answer – there’s social psychology involved. And regardless of whether PRISM has helped so far in preventing attacks it may do so in the future. Privacy, on the other hand, is fundamental and we must not be monitored by a “benign government”, regardless of the professed cause. And I genuinely believe that none of the officials involved in PRISM, envision or aim at any Orwellian dystopia, but that doesn’t mean their actions can’t lead to one. Creating the means to implement a surveillance state is just one step away from having one, regardless of the intentions that...
by Bozho | Apr 29, 2015 | Aggregated, Developer tips
As we know, meetings is where work goes to die. Discussion about the point of meetings aside, there is one type of meetings that I love. It has many names, depending on who you ask – design review, design overview, feature design. And I see it as the most important meeting in software engineering. What is it? Let’s start with a little background. You are probably “doing agile”, so you have user stories. They are already groomed and sized (based partly on some known details and partly on intuition), and you have to start working on one of them. (Below I will use the auxiliary verb “should”, because this is part description, part recommendation.) But before diving into coding, you, together with another team member, should take a step back and discuss how exactly would the implementation look like. What programming and design approaches should be taken in order to complete the story. These include discussing an API you’ll expose, the interactions between components, or even a deployment procedure. It may involve using UML on a whiteboard. Don’t waste too much time on the details or discussing “which way is slightly better than the other”, as you can refactor that later. After that you hold a meeting where you present the result of your analysis to the team. The team may have questions, or suggestions that you didn’t think about, so the meeting itself is both informative and productive. This is not applicable to all stories – some are absolutely obvious, so having a meeting would be a waste of time. But whenever there’s some specific design decision to...
by Bozho | Apr 14, 2015 | Aggregated, Developer tips, Opinions
Accidental complexity, in a broader sense, is the complexity that developers add to their code and that is not necessary for the code to work. That may include overengineering, overuse of design patterns, poor choice of tools, frameworks and paradigms, writing snippets of code in a hard to read way. For example, if you can do a project with a simple layered architecture, doing it with microservices (and having to decide on granularity, coordinate them, etc.) or message-driven architecture (with setting up the broker and its queues’ all sorts of configurations) increases the complexity of the software unnecessarily, and is therefore accidental complexity. If you want to parse XML and covert it to objects, using SAX adds a lot of accidental complexity, compared to xmo-to-object mapper (e..g JAXB), where you just add a few annotations (hopefully). If your logic can be expressed with a few lambda expressions, but instead you do several nested for-loops with if clauses inside, then that’s accidental complexity. For me at least, accidental complexity is about making it hard to read, maintain and deploy with no good reason, apart from not knowing better. Essential complexity on the other hand comes from the world you are trying to model with your software. It’s about the inevitable edge cases you have to handle if you want your software to be fit for actual use. Essential complexity can and does make your code harder to read and maintain. It makes it look like “legacy” code, but as Spolsky points out, that’s the way of things, and that’s the way it should be. Unexpected API calls, classes that exist...
by Bozho | Apr 6, 2015 | Aggregated, Developer tips
If you are running RabbitMQ in a cluster, it is not unlikely that the cluster gets partitioned (part of the cluster losing connection to the rest). The basic commands to show the status and configure the behaviour is explained in the linked page above. And when partitioning happens, you want to first be notified about that, and second – resolve it. RabbitMQ actually automatically handles the second, with the cluster_partition_handling configuration. It has three values: ignore, pause_minority and autoheal. The partitions guide linked above explains that as well (“Which mode should I pick?”). Note that whatever you choose, you have a problem and you have to restore the connectivity. For example, in a multi-availability-zone setup I explained a while ago it’s probably better to use pause_minority and then to manually reconnect. Fortunately, it’s rather simple to detect partitioning. The status command has an empty “partitions” element if there is no partitioning, and there is either a non-empty partitions element, or no such element at all, if there are partitions. So this line does the detection: clusterOK=$(sudo rabbitmqctl cluster_status | grep "{partitions,[]}" | wc -l) You would want to schedule that script to run every minute, for example. What to do with the result depends on the tool you use (Nagios, CloudWatch, etc). For Nagios there is a ready-to-use plugin, actually. And if it’s AWS CloudWatch, then you can do as follows: if [ "$clusterOK" -eq "0" ]; then echo "RabbitMQ cluster is partitioned" aws cloudwatch put-metric-data --metric-name $METRIC_NAME --namespace $NAMESPACE --value 1 --dimensions Stack=$STACKNAME --region $REGION else aws cloudwatch put-metric-data --metric-name $METRIC_NAME --namespace $NAMESPACE --value 0 --dimensions Stack=$STACKNAME --region...
by Bozho | Mar 23, 2015 | Aggregated, Developer tips, Opinions
A couple of weeks ago I asked the question “Why non-blocking?”. And I didn’t reach a definitive answer, although it seemed that writing non-blocking code is not the better option – it’s not supposed to be faster or have higher throughput, even though conventional wisdom says it should. So, leaving behind the theoretical questions, I decided to do a benchmark. The code is quite simple – it reads a 46kb file into memory and then writes it to the response. That’s the simplest scenario that’s still close the the regular usecase of a web application – reading stuff from the database, performing some logic on them, and then writing a view to the client (it’s disk I/O vs network I/O in case the database is on another server, but let’s disregard that for now) There are 5 distinct scenarios: Servlet using BIO connector, Servlet using NIO connector, Node.js, Node.js using sync file reading and Spray (a scala non-blocking web framework). Gatling was used to perform the tests, and was run on a t2.small AWS instance; the application code was run on a separate m3.large instance. The code used in the benchmark as well as the full results are available on GitHub. (Note: please let me know if you spot something really wrong with the benchmark that skews the results) What do the results tell us? That it doesn’t matter whether it’s blocking or non-blocking. Differences in response time and requests/sec (as well as the other factors) are negligible. Spray appears to be slightly better when the load is not so high, whereas BIO happens to have more errors on...
Recent Comments