aws s3/ec2/emr tips+tricks

Discussion in 'Tech Heads' started by Agrul, Nov 13, 2017.

  1. Agrul

    Agrul TZT Neckbeard Lord

    Post Count:
    47,515
    by intelligently do you mean will the cluster automatically resize itself? not with the way we have it setup now, but there is a GUI option for resizing the cluster

    i'm not sure if there's an option for asking the cluster to resize itself w/o human intervention. i havent run into anything like that but i also havent been looking for it
     
  2. Agrul

    Agrul TZT Neckbeard Lord

    Post Count:
    47,515
    i've just been shutting the cluster down at night and spinning up a new one whenever i get into work, choosing the hardware in it to suit that day's needs
     
  3. Agrul

    Agrul TZT Neckbeard Lord

    Post Count:
    47,515
  4. Jackpanel

    Jackpanel TZT Abuser

    Post Count:
    6,939
    I’m guessing amazon works the same as Azure. If so, make sure to set up an upper limit in your auto scaling rules. Otherwise you could get a big surprise on your next bill, or have everything shut down if you hit your billing cap (if you set one)
     
  5. Chemosh

    Chemosh TZT Addict

    Post Count:
    4,341
    I see this a lot. I highly suggest setting up CloudWatch metrics to alarm you via email if your bill goes over $x.00 per month. This way if you do something stupid like create a recursive AWS Lambda function and your bill ends up like 1k and you're like WTF. You can spot that shit right away.
     
  6. Chemosh

    Chemosh TZT Addict

    Post Count:
    4,341