by Bozho | Mar 19, 2020 | Aggregated, Developer tips, wordpress
WordPress is powering 35% of website. And while it may not be seen as very complex or interesting, it is one of the most prevalent technologies of our time. And many developers, even if they are not working with PHP, have to support some WordPress installation (e.g. a blog like this one). And unfortunately, there are still basic things that you can’t easily do. Plugins do help, but they don’t always have the proper functionality. Today I had to change permalinks on a website. From https://example.com/post-name to https://example.com/blog/post-name. And WordPress allows that, except there’s a problem – when you change it, the old links stop working (404). Not only that ruins your SEO, any previous share of your post will not work. I’ve pointed that out a long time ago when Spring broke their documentation links. Bottomline: you don’t want that to happen. Instead, you want to do 301 redirect (and not 302, which appears to also break your SEO). But all tutorials that are easily found online assume you can manually configure redirection through some plugin. But if you have 100-200 posts, that’s a lot of tedious work. There are plugins that allegedly monitor your posts for changes and create the redirects automatically. That may work if you manually edit a post, but it didn’t work for me when changing the permalink format settings. So how to do it in an automated way and without disrupting your website? You’d need a little bit more than a plugin – namely SQL and Regex. Here are the steps: Install and activate the Redirection plugin Open your blog’s hosting admin page...
by Bozho | Mar 13, 2020 | Aggregated, aws, Developer tips, elasticsearch, Opinions, post-mortem
So it happened that we had a production issue on the SaaS version of LogSentinel – our Elasticsearch stopped indexing new data. There was no data loss, as elasticsearch is just a secondary storage, but it caused some issues for our customers (they could not see the real-time data on their dashboards). Below is a post-mortem analysis – what happened, why it happened, how we handled it and how we can prevent it. Let me start with a background of how the system operates – we accept audit trail entries (logs) through a RESTful API (or syslog), and push them to a Kafka topic. Then the Kafka topic is consumed to store the data in the primary storage (Cassandra) and index it for better visualization and analysis in Elasticsearch. The managed AWS Elasticsearch service was chosen because it saves you all the overhead of cluster management, and as a startup we want to minimize our infrastructure management efforts. That’s a blessing and a curse, as we’ll see below. We have alerting enabled on many elements, including the Elasticsearch storage space and the number of application errors in the log files. This allows us to respond quickly to issues. So the “high number of application errors” alarm triggered. Indexing was blocked due to FORBIDDEN/8/index write. We have a system call that enables it, so I tried to run it, but after less than a minute it was blocked again. This meant that our Kafka consumers failed to process the messages, which is fine, as we have a sufficient message retention period in Kafka, so no data can be lost. I...
by Bozho | Feb 22, 2020 | Aggregated, aws, database, Developer tips, scalability
When you have to run a scalable application on AWS, your database must also be scalable. It’s easier to scale the stateless application layer, where each node is mostly disposable – even if a node in a 3-node cluster fails, you can just fire up another one and nobody notices. The database layer is stateful and therefore there’s a risk to lose data. Having just a single node is not an option, as a node can always go down and that would mean downtime. So you need multiple nodes in a cluster to make sure your application is highly available and fault tolerant (I won’t go into the differences in terminology). What database am I talking about? It doesn’t matter. It can be a SQL or a NoSQL database – each has some form of clustering available. Whether it’s active-active or active-passive. Now, for AWS in particular, you can choose RDS (or another managed option), which will handle it for you. But if there’s no managed option (e.g. Cassandra) or you don’t feel the managed option is giving you enough control, or is more expensive, or the version you require is not available, you have to manage the database layer yourself. I won’t go into the details of how to configure the database-specific clustering – you should check the documentation of the particular database for that. I’ll try to give some tips how to safely run your infrastructure that supports the database cluster. And here come auto-scaling groups. They allow you have have a group of identical nodes (based on a launch configuration) and the ASG makes sure you...
by Bozho | Feb 20, 2020 | Aggregated, Developer tips, Opinions, programming
In enterprise software the top one question you have to answer as a developer almost every day is “Where is this coming from?”. When trying to fix bugs, when developing new features, when refactoring. You have to be able to trace the code flow and to figure out where a certain value is coming from. And the bigger the codebase is, the more complicated it is to figure out where something (some value or combination of values) is coming from. In theory it’s either from the user interface or from the database, but we all know it’s always more complicated. I learned that very early in my career when I had to navigate a huge telecom codebase in order to implement features and fix bugs that were in total a few dozens line of code. Answering the question means navigating the code easily, debugging and tracing changes to the values passed around. And while that seems obvious, it isn’t so obvious in the grand scheme of things. Frameworks, architectures, languages, coding styles and IDEs that obscure the answer to the question “where is this coming from?” make things much worse – for the individual developer and for the project in general. Let me give a few examples. Scala, for which I have mixed feelings, gives you a lot of cool features. And some awful ones, like implicits. An implicit is something like a global variable, except there are nested implicit scopes. When you need some of those global variables, so just add the “implicit” keyword and you get the value from the inner-most scope available that matches the type...
by Bozho | Jan 31, 2020 | Aggregated, Developer tips, Opinions
Last month I published an article about getting email settings right. I had recently configured DKIM on our company domain and was happy to share the experience. However, the next day I started receiving error DMARC reports saying that our Office365-backed emails are failing DKIM which can mean they are going in spam. That was quite surprising as I have followed every step in the Microsoft guide. Let me first share a bit about that as well, as it will contribute to my final point. In order to enable DKIM, on a normal email provider, you just have to get the DKIM selector value from some settings screen and copy them in your DNS admin dashboard (in a new ._domainkey. record). Microsoft, instead, make you do the following: Connect to Exchange Powershell Oh, you have 2FA enabled? Sorry, that doesn’t work – follow this tutorial instead Oh, that fails when downloading their custom application? An obscure stackoverflow answer tells you that you should not download it with Firefox or Chrome – it fails to run then. You should download it with Internet Explorer (or Edge) and then it works. Just Microsoft. Now that you are connected, follow the DKIM guide In addition to powershell, you should also go in the Exchange admin in your Office365 admin, and enable DKIM Yes, you can imagine you can waste some time here. But it finally works, and you set the CNAME records and assume that what they wrote – namely, that the TXT records to which the CNAME records point, are generated on Microsoft’s end (for onmicrosoft.com), and so anyone trying to...
Recent Comments