by Bozho | Sep 29, 2018 | Aggregated, electronic identification, Opinions, security, smartcards
Electronic identity is an important concept as it lies on the crossroads of the digital, the physical and the legal worlds. How do you prove your identity without being physically present? I’ve previously given an overview talk on electronic identification, tackled the high-level and philosophical aspects, and criticized biometric-only identification as insecure. Now I want to list a few practical ways in which eID can be implemented. First, it’s important to mention the eIDAS regulation which makes electronic identity a legal reality. It says what an electronic identification scheme is and what are its legal effects (proving that you are who you claim to be). And it defines multiple “levels of assurance”, i.e. the levels of security of a method of authentication. But it is a broad framework that doesn’t tell you how to do things in the technical world. And while electronic identification is mostly cited in the context of government and municipal services, it applies to private companies as well. Currently in the US, for example, the SSN is used for electronic identification. This is a very poor approach, as leaking the SSN allows for identity theft. In the EU there are many different approaches to do that, from the Estonian PKI-based approach, to UK’s verify initiative, which relies on the databases of private companies. You can see electronic identity as a more legally-meaningful login. You still perform a login, in many cases using username and password as one of the factors, but it carries additional information – who is the actual person behind that login. In some cases it doesn’t have to even give information on...
by Bozho | Sep 24, 2018 | Aggregated, Developer tips, Opinions
You may think you have logs. Chances are, you can rely on them only for tracing exceptions and debugging. But you can’t rely on them for compliance, forensics, or any legal matter. And that may be none of your concern as an engineer, but it is one of those important non-functional requirements that, if not met, are ultimately our fault. Because of my audit trail startup, I’m obviously both biased and qualified to discuss logs (I’ve previously described what audit trail / audit logs are and how they can be maintained). And while they are a only a part of the security aspects, they are important, especially from a legal standpoint. That’s why many standards and laws – including ISO 27001, PCI-DSS, HIPAA, SOC 2, PSD2, GDPR – require audit trail functionality. And most of them have very specific security requirements. PCI-DSS has a bunch of sections with audit trail related requirements: 10.2.3 “Malicious users often attempt to alter audit logs to hide their actions, and a record of access allows an organization to trace any inconsistencies or potential tampering of the logs to an individual account. [..]” 10.5 Secure audit trails so they cannot be altered. Often a malicious individual who has entered the network will attempt to edit the audit logs in order to hide their activity. Without adequate protection of audit logs, their completeness, accuracy, and integrity cannot be guaranteed, and the audit logs can be rendered useless as an investigation tool after a compromise. 10.5.5 Use file-integrity monitoring or change-detection software on logs to ensure that existing log data cannot be changed without generating alerts...
by Bozho | Aug 21, 2018 | Aggregated, Developer tips
Recently I’ve been releasing a new build, as usual utilizing a blue-green deployment by switching the DNS record to point to the load balancer of the previously “spare” group. But before I switched the DNS, I checked the logs of the newly launched version and noticed something strange – continuous HTTP errors from our web frameworks (Spring MVC) that a certain endpoint does not support the HTTP method. The odd thing was – I didn’t have such an endpoint at all. I enabled further logging and it turned out that the request URL was not about my domain at all. The spare group, not yet having traffic directed at it, was receiving requests pointed at a completely different domain, which I didn’t own. I messaged the domain owner, as well as AWS, to inform them of the issue. The domain owner said they have no idea what that is and that they don’t have any unused or forgotten AWS resources. AWS, however, responded as follows: The ELB service scales dynamically as traffic demand changes, therefore when scaling occurs, the ELB service will take IP addresses from the AWS unused public IP address pool and assign them to the ELB nodes that are provisioned for you. The foreign domain name you see here in your case, likely belongs to another AWS customer who’s AWS resource is no longer using one of the IP addresses that your ELB node now has as it was released to the AWS unused IP pool at some stage, their web clients are very likely excessively caching DNS for these DNS names (not respecting DNS TTL),...
by Bozho | Aug 17, 2018 | Aggregated, Developer tips, jackson, JSON, performance
Sometimes you need to export a lot of data to JSON to a file. Maybe it’s “export all data to JSON”, or the GDPR “Right to portability”, where you effectively need to do the same. And as with any big dataset, you can’t just fit it all in memory and write it to a file. It takes a while, it reads a lot of entries from the database and you need to be careful not to make such exports overload the entire system, or run out of memory. Luckily, it’s fairly straightforward to do that, with a the help Jackson’s SequenceWriter and optionally of piped streams. Here’s how it would look like: private ObjectMapper jsonMapper = new ObjectMapper(); private ExecutorService executorService = Executors.newFixedThreadPool(5); @Async public ListenableFuture<Boolean> export(UUID customerId) { try (PipedInputStream in = new PipedInputStream(); PipedOutputStream pipedOut = new PipedOutputStream(in); GZIPOutputStream out = new GZIPOutputStream(pipedOut)) { Stopwatch stopwatch = Stopwatch.createStarted(); ObjectWriter writer = jsonMapper.writer().withDefaultPrettyPrinter(); try(SequenceWriter sequenceWriter = writer.writeValues(out)) { sequenceWriter.init(true); Future<?> storageFuture = executorService.submit(() -> storageProvider.storeFile(getFilePath(customerId), in)); int batchCounter = 0; while (true) { List<Record> batch = readDatabaseBatch(batchCounter++); for (Record record : batch) { sequenceWriter.write(entry); } } // wait for storing to complete storageFuture.get(); } logger.info("Exporting took {} seconds", stopwatch.stop().elapsed(TimeUnit.SECONDS)); return AsyncResult.forValue(true); } catch (Exception ex) { logger.error("Failed to export data", ex); return AsyncResult.forValue(false); } } The code does a few things: Uses a SequenceWriter to continuously write records. It is initialized with an OutputStream, to which everything is written. This could be a simple FileOutputStream, or a piped stream as discussed below. Note that the naming here is a bit misleading – writeValues(out) sounds like you are instructing...
by Bozho | Jul 29, 2018 | Aggregated, blockchain, digital signatures, Opinions, security
Recently technical and non-technical people alike started to believe that the best (and only) way to prove that something has happened in an information system is to use a blockchain. But there are other ways to achieve that that are arguably better and cheaper. Of course, blockchain can be used to do that, and it will do it well, but it is far from the only solution to this problem. The way blockchain proves that some event has occurred by putting it into a tamper-evident data structure (a hash chain of the roots of merkle trees of transactions) and distributing that data structure across multiple independent actors so that “tamper-evident” becomes “tamper-proof” (sort-of). So if an event is stored on a blockchain, and the chain is intact (and others have confirmed it’s intact), this is a technical guarantee that it had indeed happened and was neither back-dated, nor modified. An important note here – I’m stressing on “digital” events, because no physical event can be truly guaranteed electronically. The fact that someone has to enter the physical event into a digital system makes this process error-prone and the question becomes “was the event correctly recorded” rather than “was it modified once it was recorded”. And yes, you can have “certified” / “approved” recording devices that automate transferring physical events to the digital realm, e.g. certified speed cameras, but the certification process is a separate topic. So we’ll stay purely in the digital realm (and ignore all provenance use cases). There are two aspects to proving digital events – technical and legal. Once you get in court, it’s unlikely to...
Recent Comments