by Bozho | May 23, 2017 | Aggregated, Opinions
Yesterday I gave a talk that went through all the aspects of using messages queues. I’ve previously written that “you probably don’t need a message queue” – now the conclusion is a bit more nuanced, but I still stand by the simplicity argument. The talk goes through the various benefits and use cases of using message queues, and discusses alternatives of the typical “message queue broker” architecture. The slides are available here One maybe strange approach that I propose is to use distributed locks (e.g. with Hazelcast) for distributed batch processing – you lock on a particular id (possibly organization id / client id, rather than an individual record id) thus allowing multiple processors to run in parallel without stepping on each other’s toes (one node picks the first entry, the other one tries the first, but fails, and picks the second one). Something I missed as a benefit of brokers like RabbitMQ is the available tooling – you can monitor and debug your queues pretty easily. That isn’t true for simpler solutions. I wasn’t able to focus much on any of the concepts – e.g. how does brokerless work, or how to deploy a broker (e.g. RabbitMQ) in multiple data centers (availability zones), or how akka (and akka cluster) fits in the the “message queue” landscape. But I hope it’s a good overview that lets everyone have a clear picture of the options, which then they can analyze and assess in more details. The post Overview of Message Queues [slides] appeared first on Bozho's tech...
by Bozho | May 12, 2017 | Aggregated, Developer tips, Opinions
Most system have some sort of event logs – i.e. what has happened in the system and who did it. And sometimes it has a dual existence – once as an “audit log”, and once as event log, which is used to replay what has happened. These are actually two separate concepts: the audit log is the trace that every action leaves in the system so that the system can later be audited. It’s preferable that this log is somehow secured (will discuss that another time) the event log is a crucial part of the event-sourcing model where the database only stores modifications, rather than the current state. The current state is the obtained after applying all the stored modifications until the present moment. This allows seeing the state of the data at any moment in the past. There are a bunch of ways to get that functionality. For audit logs there is Hibernate Envers, which stores all modifications in a separate table. You can also have a custom solution using spring aspects or JPA listeners (that store modifications in an audit log table whenever a change happens). You can store the changes in multiple ways – as key/value rows (one row per modified field), or as objects serialized to JSON. Event-sourcing can be achieved by always inserting a new record instead of updating or deleting (and incrementing a version and/or setting a “current” flag). There some event-sourcing-native databases – Datomic and Event Store. (Note: “event-sourcing” isn’t equal to “insert-only”, but the approach is very similar) They still seem pretty similar – both track what has happened on the...
by Bozho | Feb 22, 2017 | Aggregated, digital signatures, eletronic identification, Opinions, smart cards
A smart card is a device that holds a private key securely without letting it out of its storage. The chip on your credit card is a “smart card” (yup, terminology is ambiguous – the card and the chip are interchangeably called “smart card”). There are smaller USB-pluggable hardware readers that only hold the chip (without an actual card – e.g. this one). But what’s the use? This w3c workshop from several years ago outlines some of them: multi-factor authentication, state-accepted electronic identification, digital signatures. All these are part of a bigger picture – that using the internet is now the main means of communication. We are moving most of our real-world activities online, so having a way to identify who we are online (e.g. to a government, to a bank), or being able to sign documents online (with legal value) is crucial. That’s why the EU introduced the eIDAS regulation which defines (among other things) electronic identification and digital signatures. The framework laid there is aimed at having legally binding electronic communication, which is important in so many cases. Have you ever done the print-sign-scan exercise? Has your e-banking been accessed by an unauthorized person? Well, the regulation is supposed to fix these and more more issues. Two factor authentication is another more broad concept, which has a tons of sub-optimal solutions. OTP tokens, google authenticator, sms code confirmation. All these have issues (e.g. clock syncing, sms interception, cost). There are hardware tokens like YubiKey, but they offer only a subset of the features a smart card does. But it’s not just about legally-recognized actions online and two-factor...
by Bozho | Feb 12, 2017 | Aggregated, Opinions
Sometimes it happens that people speak different languages. Even when speaking the same language. People have their own professional inclinings. Biologist may see the world as the way a cell work, cosmologist may see relationships between people as attraction between planets. And as with languages different professional afflictions give you an useful way of seeing the world. And I think everyone would find some CS concepts useful. I’ll try to list some of these concepts that I’ve found many people don’t find “native”. Primary keys – the fact that every “entity” should have a unique identifier, so that you can refer to it unambiguously. Whether it’s a UUID or an auto-increment, or a number/string derived from a special set of rules, doesn’t matter. And you may say it’s obviously, but it isn’t – I’ve seen tons of spreadsheets and registers where entities don’t have a unique identifier. Unique identifiers are useful for retrieval – if I have a driving license number, when I fill an insurance form, should I fill all the details from the driving license (name, address, age), or just a single number, and the insurer should then get the rest from a driving license database? Foreign keys (+ integrity violations and cascades) – the idea for this post came to me after I had a discussion with a bank clerk who insisted that the fact that my bank account is being deleted doesn’t mean that my virtual PoS terminal will also be deleted. To her they seemed unlinked, although the terminal is linked (via a foreign key) to the bank account. One should be able to...
by Bozho | Jan 22, 2017 | Aggregated, electronic government, Opinions
A year and a half ago I agreed to become advisor in the cabinet of the deputy primer minister of my country (Bulgaria). It might have looked like a bizarre career move, given that at the time I was a well positioned and well paid contractor (software engineer), working with modern technologies (Riak, Scala, AWS) at scale (millions of users). I continued on that project part time for a little while, then switched to another one (again part time), but most of my attention and time were dedicated to the advisory role. Since mid-December I’m no longer holding the advisory position (the prime minister resigned), but I wanted to look back, reflect and explain (to myself mainly) why that was a good idea and how it worked out. First, I deliberately continued as a part-time software engineer, to avoid the risk of forgetting my (to that point) most marketable skills – building software. But not only that – sometimes you become tired of political and administrative bullshit and just want to sit down and write some code. But the rest of my time, including my “hobby”/spare time, was occupied by meetings, research, thoughts and document drafting that aimed at improving the electronic governance in Bulgaria. I’ve already shared what my agenda was and what I was doing, and even gave a talk about our progress – opening as much data as possible, making sure we have high requirements for government software (we prepared a technical specification template for government procurement so that each administration relies on that, rather than on contractors with questionable interests), introducing electronic identification (by preparing...
by Bozho | Jan 16, 2017 | Aggregated, Opinions
UTF-8 was first presented in 1993. One would assume that 24 years is enough to time for it to become ubiquitous, especially given that the Internet is global. ASCII doesn’t even cover French letters, not to mention Cyrillic or Devanagari (the Hindi script). That’s why ASCII was replaced by ISO-8859-1, which kind of covers most of western languages. 88.3% of the websites use UTF-8. That’s not enough, but let’s assume these 11.7% do not accept any input and are just English-language static websites. The problem of the still-pending adoption of UTF-8 is how entrenched ASCII/ISO-8859-1 is . I’ll try to give a few examples: UTF-8 isn’t the default encoding in many core Java classes. Similar things go for other languages and runtimes. The default in Java is the JVM default, which is most often ISO-8859-1 (i.e. ASCII). Many frameworks, tools and containers don’t use UTF-8 by default (and don’t try to remedy the JVM not using UTF-8 by default). Tomcat’s default URL encoding I think is still ISO-8859-1. Eclipse doesn’t make the files UTF-8 by default (on my machine it’s somethings even windows-1251 (Cyrillic), which is horrendous). And so on. I’ve asked for having UTF-8 as default in the past, and I repeat my call Regex examples and tutorials always give you the [a-zA-Z0-9]+ regex to “validate alphanumeric input”. It is built-in in many validation frameworks. And it is so utterly wrong. This is a regex that must never appear anywhere in your code, unless you have a pretty good explanation. Yet, the example is ubiquitous. Instead, the right regex is [\p{L}0-9]+. Using the wrong regex means you won’t...
Recent Comments