by Bozho | Feb 17, 2016 | Aggregated, Developer tips
A pretty typical setup – spring/hibernate application that requires a distributed cache. But it turns out not so trivial to setup. You obviously need cache. There are options to do that with EhCache, Hazelcast, Infinispan, memcached, AWS’s elasticache and some others. However, EhCache supports only replicated and not distributed cache, and Hazelcast does not yet work with the latest version of Hibernate. Infinispan and Hazelcast support consistent hashing, so the entries live only on specific instance(s), rather than having a full copy of all the cache on the heap of each instances. Elasticache is AWS-specific, so Infinispann seems the most balanced option with the spring/hibernate setup. So, let’s first setup the hibernate 2nd level cache. The official documentation for infinispan is not the top google result – it is usually either a very old documentaton, or just 2 versions old documentaton. You’d better open the latest one from the homepage. Some of the options below are rather “hidden”, and I couldn’t find them easily in the documentation or in existing “how-to”s. First, add the relevant dependencies to your dependency manager configuraton. You’d need infinispan-core, infinispan-spring and hibernate-infinispan. Then in your configuratoin file (whichever it is – in my case it is jpa.xml, a spring file that defines the JPA properties) configure the following: <prop key="hibernate.cache.use_second_level_cache">true</prop> <prop key="hibernate.cache.use_query_cache">true</prop> <prop key="hibernate.cache.region.factory_class">org.hibernate.cache.infinispan.InfinispanRegionFactory</prop> <prop key="hibernate.cache.inifinispan.statistics">true</prop> <prop key="hibernate.cache.infinispan.cfg">infinispan.xml</prop> <prop key="hibernate.cache.infinispan.query.cfg">distributed-query</prop> These settings enable 2nd level cache and query cache, using the default region factory (we’ll see why that may need to be changed to a custom one later), enable statistics, point to an infinispan.xml configuraton file and change the default name for the query...
by Bozho | Feb 3, 2016 | Aggregated, Opinions
Most of us have passports, and most of these passports are by now equipped with chips that store some data, including fingerprints. But six months ago I had no idea how that operates. Now that my country is planning to roll out new identity documents, I had to research the matter. The chip (which is a smartcard) in the passports has a contactless interface. That means RFID, 13.56 MHz (like NFC). Most typical uses of smartcards require PIN entry from the owner. But the point with eMRTD (Eletronic machine readable travel documents) is different – they have to be read by border control officials anf they have to allow quickly going through Automatic Border Control gates/terminals. Typing a PIN will allegedly slow the process, and besides, not everyone will remember their PIN. So the ICAO had to invent some standard and secure way to allow gates to read the data, but at the same time prevent unauthorized access (e.g. someone “sniffing” around with some device). And they thought they did. A couple of times. First the mechanism was BAC (Basic Access Control). When you open your passport on the photo page and place it in the e-gate, it reads the machine-readable zone (MRZ) with OCR and gets the passport number, birth date and issue date from there. That combination of those is a key that is used to authenticate to the chip in order to read the data. The security issues with that are obvious, but I will leave the details to be explained by this paper. Then, they figured, they could improve the previously unsecure e-passports, and they...
by Bozho | Jan 19, 2016 | Aggregated, Opinions
A few months ago I wrote a piece in defence of monoliths and then gave a talk about it. Overall, one should not jump to microservices, because the overhead and risk are much higher than any professed benefits. But there I left out some legitimate use cases for microservices. These use cases may not be “typical” microservices, but they mostly conform to the notion of a separate, stand-alone deployment of independent functionality. The most obvious use cases are those of a CPU or RAM intensive part of the application. That normally goes into a separate deployment, offering an interface to the rest of the application. First, it’s easy to spawn multiple instances of a stateless, CPU-intensive microservice, on demand. They may even be “workers” that process a given spike and then die, including a fork-join setup. And they shouldn’t make the rest of the application get stuck because of their processing requirements – they should be separated. There are services that consume a lot of RAM (e.g. text analysis tools that include big gazetteers, trained models, natural language processing pipelines) that are impractical to be run every time a developer starts the application he’s working on. They are even problematic to redeploy and restart in a production environment. And if they change rarely, it’s justified to separate them. What’s common in those above is that they do not have a database. They expose their processing functionality but do not store anything (apart from some caching). So there is no complexity in coordinating database transactions, for example. Another “partial” use case is having multiple teams working on the same product....
by Bozho | Jan 12, 2016 | Aggregated, Opinions
I’ve written a lot about testing. Some tips on integration tests, some how-tos, some general opinions about tests. But I haven’t told my “personal story” about testing. Why are tests needed should be obvious by now. It’s not all about finding bugs (because then you can use an excuse like “QAs will find them anyway”), it’s about having a codebase that can remain stable with changes. And it’s about writing better code, because testable code is cleaner. I didn’t always write tests. Well, at least not the right amount. I had read a lot about testing, about the benefits of testing, about test-first / test-driven, about test coverage. And it seemed somewhat distant. The CRUD-like business logic seemed unworthy of testing. A few if-statements here, a few database queries there, what’s to be tested? There are companies where tests were “desirable”, “optional”, “good, but maybe not now”. There are times when marking a test with @Ignore looks ok. And although that always bites you in the end, you can’t get yourself motivated to get your coverage up. Yup, I’ve been there. I’ve written tests “every now and then”, and knew how to tests, but it wasn’t my “nature”. But I’m “clean” now – not only at work, but also in side-projects, I think I have a somehow different mentality – “how do I test that” and “how do I write that in order to be able to test it”. I won’t go into the discussion of whether “test-first” is better. I don’t do it – I’ve done it, but I don’t find it that important, provided you have the...
by Bozho | Dec 29, 2015 | Aggregated, Developer tips
Performance is a mystical thing our systems must have. But as with most thing in software engineering, there is no clearly defined set of steps that have to be followed in order to have a performant systems. It depends on the architecture, on the network, on the algorithms, on the domain problem, on the chosen technologies, on the database, etc. Apart from applying common-sense driven development, I have “collected” some general tips on how problems with performance can usually be addressed. But before that I have to make a clarification. A “performance problem” is not only about problems that you realize after you run your performance tests or after you deploy to production. Not all optimization is premature, so most of these “tips” must be applied in advance. Of course, you should always monitor, measure and try to find bottlenecks in a running system, but you should also think ahead. The first thing is using a cache. Anything that gets accessed many times but doesn’t change that often must be cached. If it’s a database table, the query should be cached. If a heavy method is invoked many times, it can be cached. Static web resources must be cached. On an algorithmic level, memoization is a useful technique. So how to do the caching? It depends. An ORM can provide the relevant cache infrastructure for database queries, spring has method-level cache support, web frameworks have resource caches. Some distributed cache (memcached/redis/ElastiCache) can be setup, but that may be too much of an effort. Sometimes it’s better and easier to have a local cache. Guava has a good cache implementation,...
Recent Comments