Five Must-Watch Software Engineering Talks

We’ve all watched dozens of talks online. And we probably don’t remember many of them. But some do stick in our heads and we eventually watch them again (and again) because we know they are good and we want to remember the things that were said there. So I decided to compile a small list of talks that I find very insightful, useful and that have, in a way, shaped my software engineering practice or expanded my understanding of the software world. 1. How To Design A Good API and Why it Matters by Joshua Bloch – this is a must-watch (well, obviously all are). And don’t skip it because “you are not writing APIs” – everyone is writing APIs. Maybe not used by hundreds of other developers, but used by at least several, and that’s a good enough reason. Having watched this talk I ended up buying and reading one of the few software books that I have actually read end-to-end – “Effective Java” (the talk uses Java as an example, but the principles aren’t limited to Java) 2. How to write clean, testable code by Miško Hevery. Maybe there are tons of talks about testing code, maybe Uncle Bob has a more popular one, but I found this one particularly practical and the the point – that writing testable code is a skill, and that testable code is good code. (By the way, the speaker then wrote AngularJS) 3. Back to basics: the mess we’ve made of our fundamental data types by Jon Skeet. The title says it all, and it’s nice to be reminded of how...

Stubbing Key-Value Stores

Every project that has a database has dilemma: how to test database-dependent code. There are several options (not mutually exclusive): Use mocks – use only unit tests and mock the data-access layer, assuming the DAO-to-database communication works Use an embedded database that each test starts and shuts down. This can also be viewed as unit-testing Use a real database deployed somewhere (either locally or on a test environment). The hard part is making sure it’s always in a clean state. Use end-to-end/functional tests/bdd/UI tests after deploying the application on a test server (which has a proper database). None of the above is without problems. Unit tests with mocked DAOs can’t really test more complex interactions that rely on a database state. Embedded databases are not always available (e.g. if you are using a non-relational database, or if you rely on RDBMS-specific functionality, HSQLDB won’t do), or they can be slow to start and this your tests may take too long supporting. A real database installation complicates setup and keeping it clean is not always easy. The coverage of end-to-end tests can’t be easily measured and they don’t necessarily cover all the edge cases, as they are harder to maintain than unit and integration tests. I’ve recently tried a strange approach that is working pretty well so far – stubbing the database. It is applicable more to key-value stores and less to relational databases. In my case, even though there is embedded cassandra, it was slow to start, wasn’t easy to setup and had subtle issues. That’s why I replaced the whole thing with an in-memory ConcurrentHashMap. Since I’m using...

We Are Not Having a Productive Debate About Women in Tech

Yes, it’s about the “anti-diversity memo”. But I won’t go into particular details of the memo, the firing, who’s right and wrong, who’s liberal and who’s conservative. Actually, I don’t need to repeat this post, which states almost exactly what I think about the particular issue. Just in case, and before someone decided to label me as “sexist white male” that knows nothing, I guess should clearly state that I acknowledge that biases against women are real and that I strongly support equal opportunity, and I think there must be more women in technology. I also have to state that I think the author of “the memo” was well-meaning, had some well argued, research-backed points and should not be ostracized. But I want to “rant” about the quality of the debate. On one side we have conservatives who are throwing themselves in defense of the fired googler, insisting that liberals are banning conservative points of view, that it is normal to have so few woman in tech and that everything is actually okay, or even that women are inferior. On the other side we have triggered liberals that are ready to shout “discrimination” and “harassment” at anything that resembles an attempt to claim anything different than total and absolute equality, in many cases using a classical “strawman” argument (e.g. “he’s saying women should not work in tech, he’s obviously wrong”). Everyone seems to be too eager to take side and issue a verdict on who’s right and who’s wrong, to blame the other side for all related and unrelated woes and while doing that, exhibit a huge amount of...

How To Send Ethereum Transactions With Java

After I’ve expressed my concerns about the blockchain technology, let’s get a bit more practical with the blockchain. In particular, with Ethereum. I needed to send a transaction with Java, so I looked at EthereumJ. You have three options: Full node – you enable syncing, which means the whole blockchain gets downloaded. It takes a lot of time, so I abandoned that approach “Light” node – you disable syncing, so you just become part of the network, but don’t fetch any parts of the chain. Not entirely sure, but I think this corresponds to the “light” mode of geth (the ethereum CLI). You are able to send messages (e.g. transaction messages) to other peers to process and store on the blockchain, but you yourself do not have the blockchain. Offline (no node) – just create and sign the transaction, compute its raw representation (in the ethereum RLP format) and push it to the blockchain via a centralized API, e.g. the etherscan.io API. Etherscan is itself a node on the network and it can perform all of the operations (so it serves as a proxy) Before going further, maybe it’s worth pointing out a few general properties of the blockchain (the ethereum one and popular cryptocurrencies at least) – it is a distributed database, relying on a peer-to-peer (overlay) network, formed by whoever has a client software running (wallet or otherwise). Transactions are in the form of “I (private key owner) want to send this amount to that address”. Transactions can have additional data stored inside them, e.g. representing what they are about. Transactions then get verified by peers (currently...

Concerns About The Blockchain Technology

The so-called (and marketing-branded) “blockchain technology” is promised to revolutionize every industry. Anything, they say, will become decentralized, free from middle men or government control. Services will thrive on various installments of the blockchain, and smart contracts will automatically enforce any logic that is related to the particular domain. I don’t mind having another technological leap (after the internet), and given that I’m technically familiar with the blockchain, I may even be part of it. But I’m not convinced it will happen, and I’m not convinced it’s going to be the next internet. If we strip the hype, the technology behind Bitcoin is indeed a technical masterpiece. It combines existing techniques (likes hash chains and merkle trees) with a very good proof-of-work based consensus algorithm. And it creates a digital currency, which ontop of being worth billions now, is simply cool. But will this technology will be mass-adopted, and will mass adoption allow it to retain the technological benefits it has? First, I’d like to nitpick a little bit – if anyone is speaking about “decentralized software” when referring to “the blockchain”, be suspicious. Bitcon and other peer-to-peer overlay networks are in fact “distributed” (see the pictures here). “Decentralized” means having multiple providers, but doesn’t mean each user will be full-featured nodes on the network. This nitpicking is actually part of another argument, but we’ll get to that. If blockchain-based applications want to reach mass adoption, they have to be user friendly. I know I’m being captain obvious here (and fortunately some of the people in the area have realized that), but with the current state of the technology,...

Basic API Rate-Limiting

It is likely that you are developing some form of (web/RESTful) API, and in case it is publicly-facing (or even when it’s internal), you normally want to rate-limit it somehow. That is, to limit the number of requests performed over a period of time, in order to save resources and protect from abuse. This can probably be achieved on web-server/load balancer level with some clever configurations, but usually you want the rate limiter to be client-specific (i.e. each client of your API sohuld have a separate rate limit), and the way the client is identified varies. It’s probably still possible to do it on the load balancer, but I think it makes sense to have it on the application level. I’ll use spring-mvc for the example, but any web framework has a good way to plug an interceptor. So here’s an example of a spring-mvc interceptor: @Component public class RateLimitingInterceptor extends HandlerInterceptorAdapter { private static final Logger logger = LoggerFactory.getLogger(RateLimitingInterceptor.class); @Value("${rate.limit.enabled}") private boolean enabled; @Value("${rate.limit.hourly.limit}") private int hourlyLimit; private Map<String, Optional<SimpleRateLimiter>> limiters = new ConcurrentHashMap<>(); @Override public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception { if (!enabled) { return true; } String clientId = request.getHeader("Client-Id"); // let non-API requests pass if (clientId == null) { return true; } SimpleRateLimiter rateLimiter = getRateLimiter(clientId); boolean allowRequest = limiter.tryAcquire(); if (!allowRequest) { response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value()); } response.addHeader("X-RateLimit-Limit", String.valueOf(hourlyLimit)); return allowRequest; } private SimpleRateLimiter getRateLimiter(String clientId) { if (limiters.containsKey(clientId)) { return limiters.get(clientId); } else { synchronized(clientId.intern()) { // double-checked locking to avoid multiple-reinitializations if (limiters.containsKey(clientId)) { return limiters.get(clientId); } SimpleRateLimiter rateLimiter = createRateLimiter(clientId); limiters.put(clientId, rateLimiter); return rateLimiter; } } } @PreDestroy public void...