High security networks

Discussion in 'Tech Heads' started by Solayce, Jan 25, 2019.

  1. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    Anyone have experience here, either on one, or building/maintaining one?

    The discussions are currently around 3 tiered access layers: Open (Business functions), Firewalled (HIPAA compliant), and possibly air-gapped.

    In such a scenario, are smart cards still the best/only solution or are there better options out there?
     
  2. Utumno

    Utumno Administrator Staff Member

    Post Count:
    41,193
    I have some input here, but am pretty strapped for time. I also work with an excellent security guru who could answer this shit much better than me. Will try to get back.

    What is the big picture here, you obviously already have a network already w/some sort of security model (even if the model is shit or just organically grown).

    Are you trying to revamp your network? Lock it down more?
     
  3. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    Sort of.

    ============
    Part 1 - Main job
    ============

    Current org, is a little bit country and a little bit rock'n'roll, w.r.t. university and corporate networks. Basically, it's organically grown shitty. We have a Faculty with titles matching along with that, Professor, Assistant Professor, Post Doc, Research Associate, and etc, but we are not an educational org. We're nonprofit basic research, with most Faculty hires coming from Universities and a few home grown Post Docs becoming Faculty.

    Our research has never directly included humans, and therefore never requiring HIPAA compliance, so we didn't have any compliance requirements until about 8 or so years ago when we started needing some minimum FISMA requirements that the Gov't put in place for federal research. FISMA has several levels, of which we're the lowest, but the higher you go up, the more/better grants you can apply for.

    So, in general, I have been trying to move us into a more compliant direction to be able to apply for bigger grants. In practice, that has simply meant upgrading AD, Exchange, and our PKI. I plan to deploy LAPS in the next month or two, while finishing the Exchange upgrade from 2007->2013; then hopefully a quick jump to 2016 (or 2019 if it exists at that time).

    I have also segregated our admin accounts further. Each admin used to have a user account and an admin account, but I have added extra accounts for Tiering - user account, workstation admin account, server admin account, domain admin account, enterprise admin account. This + LAPS is Microsft's recommendation to move towards a "Red Forest" design, aka ESAE:

    https://social.technet.microsoft.co...security-administrative-environment-esae.aspx

    Up until this point, our current data footprint is about 500TB compressed on a Data Domain, for everything (VM backups, as well as project data and warm archive data). We have consolidated down to one Isilon cluster (hybrid SSD/HDD) for project and departmental data, that also serves as scratch to our HPC Grid. We still use tapes for long term backups (monthlies to tape for 6 months with occasional cold data for 5 years, by request only), but have, in the last year or so, been deploying some home-grown ZFS clusters to get off of tape in the future, hopefully.

    HPC has traditionally been filled with old VMware compute Hosts (Dell r900s: 4 socket, 1TB RAM each) using LDAP as directory service and most data shared over NFS.

    ===============
    Part 2 - New Venture
    ===============

    In the current scenario, and where I would need the most guidance, the Org is looking to start a new venture. It will be dealing with cutting edge AI/ML research, but focused completely on humans; possibly many, many, humans. It has been repeated to me that this will most likely spin off from us (nonprofit) to a for profit venture, if things go well. So, somehow, we need to kick this thing off with designs that meet the minimum requirements to start off, but to be able to scale from Terabyte/Petabyte to potentially Exabyte scale.

    The design request, due to the large spinoff probability, is for this to be completely separate infrastructure: new forest(s), domain, O365 tenant, whatever is needed. Compliance needs are HIPAA requirements. Network to contain at least 2 access tiers and the possibility of an air-gapped network, too. I don't have a lot of direction beyond that, other than the AI/ML requirements will be focused on the human data; unknown limit, but right now I have heard each person's data already reaches 200GB, but that could get bigger, depending on the associate data that will get included. Assume the data set will contain full medical history (text) including various blood and machine tests, as well as DNA sequencing data, to be all stitched together, analyzed, and cross-referenced across the whole data set. Beginning sample to be about 50 people, ultimately leading to very, very large numbers of people.

    I assume PPI will live in the air-gapped network, the majority of research done on de-identified data in the HIPAA tier, and day-to-day business (where I usually work), in the open area.

    If I wasn't clear, or you have more questions, ask away, and I'll answer what I can, if I even know.
     
  4. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    adding a thread watch so I can more efficiently check thread. TIA for any additional info. And really, quality resources to read is the most I'm hoping to get here. Anything more is bonus.
     
  5. Czer

    Czer I'm a poor person. The lambo is my cousin's.

    Post Count:
    24,939
    I bounce between sysadmin/network engineering but I do devops/implementation(consultation)/dev support

    i've never worked in a grid environment so I had to look up a lot of that stuff

    so are you asking how to connect that together or are you talking about what types of admin systems and security required

    but beware software like arcsight's base code was leaked to russia in 2017 and probably most security software
     
  6. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    Maybe a little bit of both. Part 1 and Part 2 shouldn't have to communicate. This is mostly about Part2.

    How would you design a greenfield deployment with those requirements?
     
  7. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    Also, the thought occurred to me last night that the air-gapped network may be bigger than I expected, since - once again I am assuming - that most doctor/patient communication will happen there, as well as labs and etc.
     
  8. Czer

    Czer I'm a poor person. The lambo is my cousin's.

    Post Count:
    24,939
    as far as air gapping, you need something that will watch removable drives, and I assume people will not be allowed to carry their electronics into this area I.e. you need an ice box for people entering to drop off their actual stuff?

    do you want to be tempest compliant etc

    https://en.wikipedia.org/wiki/Tempest_(codename) aka tempested buildings
     
    Last edited: Jan 27, 2019
  9. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    That can be done by group policy in windows and/or profiles in macOS. As for tempest, I'm not sure. I wouldn't rule this out. It may be starting as a project with armed forces.

    And thanks for the link. Hadn't heard of that, so references are good. We will probably behaving CT Scanner and/or MRI at the location, which will have its own isolation requirements, but I hadn't considered EM.
     
  10. Utumno

    Utumno Administrator Staff Member

    Post Count:
    41,193
    This is probably not the answer you are looking for, but given the depth of what you have already posted - my gut feeling tells me you should get a security person hired full time to drive this. I think certain compliance creds (SOC2 for example) actually require a CISO - I feel like you are approaching this like you've always approached everything (which is to read everything you can about it and try to do it all yourself to save your company a buck).

    I know that's worked for you thus far career-wise, but I'd bet almost anything you're underpaid and overworking yourself. To be completely cynical about it, from what I've seen - most security practices and implementation w/regards to widely-accepted standards SOC2, HIPPA, ISO, etc. are more about getting all your paperwork in order, checking all the boxes, and maintaining this stuff over time (v.s. actually spending a lot of time thinking about how secure your network actually is from hostile actors). That is not to say that the controls put in place from those standards are useless! Often it is a perfect driver to get a business to take security seriously and implement much needed protections.

    But the main hurdle I've seen in compliance is simply time (and not technology or know-how). It's someone sitting down and documenting the *fuck* out of everything, while working with partners/auditors who won't gouge the shit out of you, and are ethical enough to really audit your processes, while not being complete pain in the asses about nitpicking every little thing.
     
  11. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    My Title is Senior Manager IT. In practice I'm a Senior Systems Engineer, with oversight of all end user machines, all windows and Mac servers (~80), Enterprise apps (Exchange/O365, MS SQL Server), sharing support of our VMware cluster until we're completely on Nutanix, leading a team of 4 junior Admins, responsible for a 300K budget. I am within normal bounds of salary for the area. I'm maybe closer to, or working towards, this:

    ======================
    Enterprise Operations Director

    Also referred to as: Director IT Infrastructure Operations


    Requirements and Responsibilities
    Directs information systems operations and for software systems analysis and programming. Monitors accessibility of applications or services. Leads implementation and maintenance of enterprise-wide system tools and ensures scalability. Implements and maintains operating policies to mitigate risk. May also be responsible for information center, database management, data security, telecommunications, or information systems training. Requires a bachelor's degree. Typically reports to top management. Manages a departmental sub-function within a broader departmental function. Creates functional strategies and specific objectives for the sub-function and develops budgets/policies/procedures to support the functional infrastructure. Deep knowledge of the managed sub-function and solid knowledge of the overall departmental function. Typically requires 5+ years of managerial experience.
    ========================

    In which case, I'm significantly underpaid, but should get a Director promotion in the next couple years. I already report directly to CTO.
     
  12. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    There was a Security consultant hired recently, who only just this week showed up. If she's temporary, or doesn't pan out, the clock is ticking to get something started, so I am hoping to fulfill that initial, skeleton, deployment
     
  13. Agrul

    Agrul TZT Neckbeard Lord

    Post Count:
    45,824
    Any reqs re: encryption for all on-disk and in-transmission data for HIPAA protections?
     
  14. Agrul

    Agrul TZT Neckbeard Lord

    Post Count:
    45,824
    (if there aren't, you should do so anyway)
     
    Solayce likes this.
  15. Utumno

    Utumno Administrator Staff Member

    Post Count:
    41,193
    Are you actually the direct supervisor for those 4? If so, I rest my case (about you doing too much). By the time you're actually running the show at that level you should barely be doing any IC work anymore and delegating everything possible.

    Good to hear about the security consultant though - hopefully she's good! It's unfortunately hard to find good infosec people, but very worthwhile to get someone on staff. Alternatively, if you have someone who is responsible, handles vendors well and has the right mindset, you can sometimes grow one of your own admins into a security person (they have to be pretty strong on the people-skills side though, which is also kindof rare).
     
  16. Solayce

    Solayce Would you like some making **** BERSERKER!!! Staff Member

    Post Count:
    21,654
    If not everywhere, then certainly in the HIPAA and air-gapped tier. Not sure how that effects HPC though. Thanksfully not my responsibility.

    The 4 are on 2 coasts and are basically entry level sys admins - well 3 are. I have a level2 that is almost where he needs to be w.r.t. mastering all helpdesk type stuff and ready to move to sa3, which is what I consider server/application support. I rarely do helpdesk stuff anymore, but the team is not experienced enough to research, design and deploy anything. They basically don't know what they don't know. The SA2 is finally understanding thee gravity/enormity of what an SA3 will be. We do already have an InfoSec person, but I would consider him and entry level infosec. This is far and above any responsibility he's ever had and he hasn't mastered/rewritten our tiny requirements yet; though that's more about his workload and who he reports to.
     
  17. Agrul

    Agrul TZT Neckbeard Lord

    Post Count:
    45,824
    depends on the kind of HPC. it's also relevant to simple stuff like amazon machine images too, though -- by default they do not encrypt their boot volumes, have to put together a custom AMI to get them to encrypt properly. for everythign other than the boot volume it's generally simpler, just finding the right parameter settings to force on in the startup sequence for EBS encryption, the several spark encryption options, etc

    if they're working directly with large-scale message-passing algorithms though then ease of encryption's probably a lot more dependent on whatever custom libraries or code they're working w/
     
    Solayce likes this.
  18. Czer

    Czer I'm a poor person. The lambo is my cousin's.

    Post Count:
    24,939
    agrul can you give a description of their kind of network and the necessary things they would need to secure it
     
  19. Agrul

    Agrul TZT Neckbeard Lord

    Post Count:
    45,824
    no -- not a networking security person. my encryption suggestions were more about individual machines in common cloud environments (e.g. boot vol encryption on EC2 instances or in EMR nodes by creating new AMIs: https://aws.amazon.com/blogs/aws/new-encrypted-ebs-boot-volumes/) than about general network security. the only "network security" suggestions i would have are to make sure the encryption options in any high-performance computing software are turned on, so that e.g. spark shuffles that involve large data transfers between machines in an EMR cluster are properly encrypted. spark has a bunch of these that are pretty well documented: https://www.cloudera.com/documentation/enterprise/5-8-x/topics/sg_spark_encryption.html
     
  20. Czer

    Czer I'm a poor person. The lambo is my cousin's.

    Post Count:
    24,939
    Thank you.

    I'm going to read over that.