by Bozho | Apr 17, 2020 | Aggregated, alarms, alerts, aws, Developer tips, devops
Monitoring is key for any real-world application. You have to know what’s happening and be alerted in real time if something wrong is happening. AWS has CloudWatch for that, and gives you a lot of metrics automatically. But there are some that you have to define yourself. And then you need to define proper alarms. Here I’ll focus on hour: High number of application errors High number of application warnings High number of 5xx errors on the load balancer High number of 4xx errors on the load balancer First, the prerequisites: You need to be using CloudFormation to automate everything. You can create all of those things manually, but automation is a big plus If using CloudFormation, you’d preferably have a sub-stack for configuring alarms You need to be collecting your logs with CloudWatch logs If you are not using CloudWatch logs, here’s a simple config file and script to enable them: { "agent": { "metrics_collection_interval": 10, "region": "eu-west-1", "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "logs": { "logs_collected": { "files": { "collect_list": [ { "file_path": "{{logPath}}", "log_group_name": "{{logGroupName}}", "log_stream_name": "{instance_id}", "timestamp_format": "%Y-%m-%d %H:%M:%S" } ] } } } } # install AWS CloudWatch monitor mkdir cloud-watch-agent cd cloud-watch-agent wget https://s3.amazonaws.com/amazoncloudwatch-agent/linux/amd64/latest/AmazonCloudWatchAgent.zip unzip AmazonCloudWatchAgent.zip ./install.sh aws s3 cp s3://$BUCKET_NAME/cloudwatch-agent-config.json /var/config/cloudwatch-agent-config.json sed -i -- 's|{{logPath}}|/var/log/application.log|g' /var/config/cloudwatch-agent-config.json sed -i -- 's|{{logGroupName}}|app_node|g' /var/config/cloudwatch-agent-config.json sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/var/config/cloudwatch-agent-config.json -s Now you have to define two things: Log metrics and alarms. The cloudformation code below creates both: "HighAppErrorsNotification": { "Type": "AWS::CloudWatch::Alarm", "Properties": { "AlarmActions": [ { "Ref": "NotificationTopicId" } ], "InsufficientDataActions": [ { "Ref": "NotificationTopicId" } ], "AlarmDescription": "Notify if there are too many application...
by Bozho | Mar 12, 2017 | Aggregated, Developer tips, devops, encryption, security
If you are building a service that stores sensitive data, your number one concern should be how to protect it. What IS sensitive data? There are some obvious examples, like medical data or bank account data. But would you consider a dating site database as sensitive data? Based on a recent leaks of a big dating site I’d say yes. Is a cloud turn-by-turn nagivation database sensitive? Most likely, as users journeys are stored there. Facebook messages, emails, etc – all of that can and should be considered sensitive. And therefore must be highly protected. If you’re not sure if the data you store is sensitive, assume it is, just in case. Or a subsequent breach can bring your business down easily. Now, protecting data is no trivial feat. And certainly cannot be covered in a single blog post. I’ll start with outlining a few good practices: Don’t dump your production data anywhere else. If you want a “replica” for testing purposes, obfuscate the data – replace the real values with fakes ones. Make sure access to your servers is properly restricted. This includes using a “bastion” host, proper access control settings for your administrators, key-based SSH access. Encrypt your backups – if your system is “perfectly” secured, but your backups lie around unencrypted, they would be the weak spot. The decryption key should be as protected as possible (will discuss it below) Encrypt your storage – especially if using a cloud provider, assume you can’t trust it. AWS, for example, offers EBS encryption, which is quite good. There are other approaches as well, e.g. using LUKS with keys...
Recent Comments