dbag.tech

chaos advocate

resume update

by Ryan Gravlin on

A recent change in my life has required me to update my resume. As such I figured it would be a fun time to use what I’ve learned to create something for myself that I can use now and in the future. I’ve completed a couple of iterations and am comfortable where it’s at right now. I plan future revisions and features that will be completed as time permits.

Ryan Gravlin’s Terraform Resume Runner

It’s been great fun so far and I have learned quite a bit! A small list of tools I used:

  • Windows 10 Home (no native virtualization required me to use Vagrant instead of native Docker)
  • Vagrant (created Vagrant files to rapidly create local build and deployment enviroments)
  • VirtualBox (provider for Vagrant)
  • Ubuntu 18.04 (used as base build image)
  • Microk8s (used for local testing)
  • Terraform (used for AWS resource creation)
  • Ruby using Sinatra framework and Thin webserver (used for application frontend)
  • RubyMine with remote interpreter (used for code quality)
  • LaTeX
  • AWS Resources (defined by Terraform)

You can activate the application by running the following command from a terminal:

$> curl -s https://dbag.tech/resume

Remember that there is no guarantee what action will be run when you run the above command. The logic flips between apply and destroy after successful runs. Please run the command again to destroy the resources after you create them!

If you contact me with your hexadecimal namespace I can also tell you exactly how much your infrastructure cost me!

Please visit the GitHub link above to read the code and see upcoming features I plan!

read more

Elastic Stack v6 Overview

by Ryan Gravlin on

The Inputs:

  • Docker Container STDOUT/STDERR
    • >1 process container logs
  • EC2 Instances
    • System Logs
    • Application Logs
  • AWS CloudWatch
  • AWS Kinesis

The Problems:

  • Standardization
  • Centralized
  • Automated
  • Agents
  • Compliance
  • Guarantees
  • Fast Retrieval
  • Archived Storage

The Solutions:

  • Storage: ElasticSearch and S3
  • Visualize: Kibana
  • Manage: Curator
  • Queue Management: LogStash
  • Log Management: FileBeat
  • Metric Collection: MetricBeat

read more

it is done

by Ryan Gravlin on

I have completed the respondinator project to my liking, which is rare. I have been able to squash all issues that have come up, protect it against specific types of attacks, and clean up the code to my liking. I added integration tests to make sure I don’t break anything, and I have pushed the changes to a public GitHub repository. You can find the code base at the following location:

Below will be the final instructions moving foward until any changes are made:

Adding routes (POST)

String response route:

$> curl -s -d '{"path": "/hello", "response": "world"}' https://dbag.tech/routes/addme
$> curl -s https://dbag.tech/routes/hello
world

JSON response route:

$> curl -s -d '{"path": "/hello", "response": { "world": { "hello": "world" } }}' https://dbag.tech/routes/addme
$> curl -s https://dbag.tech/routes/hello
{"world":{"hello":"world"}}

Respondinator response:

{
  "path": "/hello",
  "response": "world",
  "key": "b08421d1-4cd5-4668-96d2-bf1467430db5"
}

Updating routes (PUT)

  • Use the key that was provided in the POST or PUT
$> curl -s -XPUT -d '{"path": "/hello", "response": { "world": { "hello": "dbag" } }, "key": "b08421d1-4cd5-4668-96d2-bf1467430db5"}' https://dbag.tech/routes/addme
$> curl -s https://dbag.tech/routes/hello
{"world":{"hello":"dbag"}}

Retrieving route (GET)

$> curl -s https://dbag.tech/routes/hello | jq .
{
  "world": {
    "hello": "dbag"
  }
}

Additional NGINX update

This issue was brought to my attention by a specially crafted Go application that my associate built specifically to take down my application. I was able to prevent further attacks by this application by modifying my base NGINX configuration to rate limit requests. Due to the fact that I use CloudFront in front of my origin, I had one of two choices. Use the binary representation of the IP address in NGINX to block the requests (less memory usage), or block based on X-Forwarded-For address (more memory usage). I chose the former, due to the fact that these applications are running in memory constricted containers.

What happened before adding the set_real_ip_from options was that it was rate limiting each CloudFront edge IP address instead of the true client IP. After setting these options, I no longer see the CloudFront IP address, and all IPs are replaced with the true client IP. I’m still not sure that I want to keep it this way, as it may be useful at some point to know the CloudFront edge IP that’s hitting the site.

NGINX HTTP Context:

    # Rate limiting
    limit_req_zone	$binary_remote_addr zone=mylimit:10m rate=5r/s;

    # Real-IP Module for CloudFront
    real_ip_header	X-Forwarded-For;
    real_ip_recursive	on;

    # CloudFront IP Ranges
    set_real_ip_from	13.32.0.0/15;
    set_real_ip_from	13.54.63.128/26;
    set_real_ip_from	13.59.250.0/26;
    set_real_ip_from	13.113.203.0/24;
    set_real_ip_from	13.124.199.0/24;
    set_real_ip_from	13.228.69.0/24;
    set_real_ip_from	34.195.252.0/24;
    set_real_ip_from	34.226.14.0/24;
    set_real_ip_from	34.232.163.208/29;
    set_real_ip_from	35.158.136.0/24;
    set_real_ip_from	35.162.63.192/26;
    set_real_ip_from	35.167.191.128/26;
    set_real_ip_from	52.15.127.128/26;
    set_real_ip_from	52.46.0.0/18;
    set_real_ip_from	52.47.139.0/24;
    set_real_ip_from	52.52.191.128/26;
    set_real_ip_from	52.56.127.0/25;
    set_real_ip_from	52.57.254.0/24;
    set_real_ip_from	52.66.194.128/26;
    set_real_ip_from	52.78.247.128/26;
    set_real_ip_from	52.84.0.0/15;
    set_real_ip_from	52.199.127.192/26;
    set_real_ip_from	52.212.248.0/26;
    set_real_ip_from	52.220.191.0/26;
    set_real_ip_from	52.222.128.0/17;
    set_real_ip_from	54.182.0.0/16;
    set_real_ip_from	54.192.0.0/16;
    set_real_ip_from	54.230.0.0/16;
    set_real_ip_from	54.233.255.128/26;
    set_real_ip_from	54.239.128.0/18;
    set_real_ip_from	54.239.192.0/19;
    set_real_ip_from	54.240.128.0/18;
    set_real_ip_from	204.246.164.0/22;
    set_real_ip_from	204.246.168.0/22;
    set_real_ip_from	204.246.174.0/23;
    set_real_ip_from	204.246.176.0/20;
    set_real_ip_from	205.251.192.0/19;
    set_real_ip_from	205.251.249.0/24;
    set_real_ip_from	205.251.250.0/23;
    set_real_ip_from	205.251.252.0/23;
    set_real_ip_from	205.251.254.0/24;
    set_real_ip_from	216.137.32.0/19;

NGINX Server Context:

    location /routes/ {
      limit_req				zone=mylimit burst=20 nodelay;
      limit_req_status			429;
      proxy_pass			http://respondinator/;
      proxy_set_header Host		$host;
      proxy_set_header X-Real-IP	$remote_addr;
    }

read more

update to insanity

by Ryan Gravlin on

I have updated The Respondinator. You’ll find examples on how to interact below, and quick and dirty summary.

  • POST is a creation method, and only requires path/response (will reject routes that already exist)
  • PUT is an update method, and requires path/response/key (will reject payloads with invalid keys or if it doesn’t have the key)
  • Response can be valid JSON or string

How to ADD a route

  • HTTP POST to https://dbag.tech/routes/addme with a JSON payload that contains a path and response key

Example string:

$> curl -s -d '{"path": "/hello", "response": "world"}' https://dbag.tech/routes/addme

Example JSON:

$> curl -s -d '{"path": "/hello", "response": { "world": { "hello": "world" } }}' https://dbag.tech/routes/addme

Response:

{
  "path": "/hello",
  "response": "{\"key\":{\"subkey\":\"value\"}}",
  "key": "b08421d1-4cd5-4668-96d2-bf1467430db5"
}

How to UPDATE a route

  • HTTP PUT to https://dbag.tech/routes/addme with a JSON payload that contains a path, response, and key
$> curl -s -XPUT -d '{"path": "/hello", "response": { "world": { "hello": "dbag" } }, "key": "b08421d1-4cd5-4668-96d2-bf1467430db5"}' https://dbag.tech/routes/addme

How to GET a route

  • HTTP GET to https://dbag.tech/routes/:route
$> curl -s https://dbag.tech/routes/hello | jq .

{
  "world": {
    "hello": "dbag"
  }
}

read more

the respondinator

by Ryan Gravlin on

In an effort to learn more about Ruby and APIs I came up with the idea of an API that dynamically adds routes to itself, and will respond with a payload that is specified by the user. The use case for this that I thought of was in the event you are going to interact with a 3rd party API, you would be able to build a quick request/response mockup to test against without actually hitting the 3rd party. You can create invalid responses to test your error handling, or mess with the data, whatever your heart desires.

The current iteration consumes a JSON payload with up to three keys:

curl -s -d '{"path": "/hello", "response": "world"}' https://dbag.tech/routes/addme
{
  "path": "/hello",
  "response": "world"
}

You will receive a response similar to:

{
  "path":"/hello",
  "response":"world",
  "key":"67ba07a5-e1e7-43ae-b3d2-a8709e09c117"
}

You can then update the route using the key provided.

curl -s -d '{"path": "/hello", "response": "dbag", "key": "67ba07a5-e1e7-43ae-b3d2-a8709e09c117"}' https://dbag.tech/routes/addme

I only currently respond with a string. The idea is to be able to consume an entire JSON response as well as allowing a POST with a response. It’s almost entirely useless right now.

read more