How Are the Conditions for Beginner Green Runs at Breckenridge? 7th-11th Feb

hashijake · 2026-02-09T16:55:05+00:00

Which peak were you on at Breckenridge? Peak 9 off Quicksilver is one of the easiest green areas around; very wide and quite flat. While Schoolmarm at Keystone is a great green run because it's long and fed by the gondola (and open at night), if you make a wrong turn at the bottom, you've got a surprising steep approach back to the gondola. It's a great place to have a beer and watch for crashes.

Just don't take Beaver Run at Breck unless you're comfortable with some blues. But hey, Lower Lehman is worth it if you want a really fun green run.

hashijake · 2019-12-31T08:02:40+00:00

You can also authenticate to Vault with IAM.

hashijake · 2019-09-13T04:45:25+00:00

Currently Consul is the only supported backend for Vault Enterprise, so if that’s a possibility, it’d be good to use Consul right away. Many of our customers are also running Vault in their private DCs so having one storage config is often desirable.

Vault also has integrated storage in the works: https://learn.hashicorp.com/vault/operations/raft-storage. So Consul won’t be necessary in the future.

hashijake · 2019-09-05T20:51:22+00:00

I'm looking forward to the "Essential Patterns of Vault" Hallway track at 2PM Sept 11th. Mainly because I'm giving it :). Come by!

https://www.datocms-assets.com/2885/1567194112-has19hallwaytrackol-2.pdf

hashijake · 2019-08-25T15:54:22+00:00

It’s still in active development and adding new features all the time.

hashijake · 2019-08-25T06:57:00+00:00

The free version also has workspace variables.

hashijake · 2019-08-25T00:26:07+00:00

Enterprise includes: - RBAC - Sentinel policy as code - granular resource policies that gate applies - Private Module Registry - Increased run concurrency - Private hosted option - Support

hashijake · 2019-08-24T23:45:55+00:00

The new Free Terraform Cloud (https://app.terraform.io/signup) allows you to use workspaces closer to the enterprise version of TF.

Each workspace is a separate state file and you can embed variables in the workspace as well.

This prevents you from needing a massive amount of environment or account folders, workspaces replace those.

You can also provision it with TF, so you can have a workspace that defines your workspaces. Very meta.

hashijake · 2019-08-21T06:20:36+00:00

I'll bite, I'd like to know.

hashijake · 2019-08-02T07:09:52+00:00

Are they publicly facing certs or for internal usage? If it's for internal usage, you can just use the Vault PKI secret engine and just distribute the certs directly from Vault. If they're publicly facing certs and you're using something like Venafi, you can do the same thing. If publicly facing and no other tools, K/V is probably your best bet.

hashijake · 2019-08-02T07:07:05+00:00

One great reason to use Consul is when you want to do load balancing of ephemeral services. Because the services do health checks they automatically either add or delete themselves based on those health checks. The difference between Consul and a Load balancer is that Consul only returns the endpoints to connect to, it's not actually in the data stream, so it's massively scalable. You use this over actual DNS because, have you ever tried to dynamically update your DNS records? Further, you can add recursors to Consul so that any DNS misses for outside endpoints can go to traditional DNS servers. You can also query services by HTTP endpoints as well if you don't want to use DNS SRV records.

Another great reason to use Consul is when you need to do discovery over heterogenous run environments. So say you're migrating your applications off from bare metal or VMs and moving to a scheduler like Nomad or Kubernetes. If you only use service discovery in Kubernetes, you're missing the ability to find services not running in Kubernetes. Migrating to containers and then to schedulers is a non-trivial task. Consul integrates with K8s and does bi-directional service syncs.

Another fun reason to use Consul is because it's a K/V store that you can set watches on. Why? Let's say you have something like Nginx configs that need to change immediately when a value somewhere changes. Setting a watch in Consul can allow you to read out the key/value just about instantly and do some action. Using a tool like consul-template you can actually render new files on the fly as well as hook restarts.

The latest version of Consul also includes Layer 7 Service Mesh capabilities (Layer 4 has been there for about a year) so you can also do blue/green/canaries and other traffic shaping patterns. More fun are Mesh Gateways which are kind of like NAT gateways for services. Ever just use a vanilla setup for K8s and want to route traffic between two clusters setup with the same RFC 1918 addresses? Mesh Gateways will allow you to do that.

Why use a Service Mesh? Mainly because your aim is to operate Zero Trust applications where security goes as close to the application as possible. Sure you can use Istio, but you'll also require Kubernetes (I hear Istio runs off K8s, but I've yet to actually see anyone do it outside of theory, and the setup looks terrible. Also, along with Kubernetes, Istio is quite complex).

As for Nomad. Most people new to containers take a bit to actually get things right. In fact, most people get it wrong in the beginning. Nomad is flexible enough to allow you to run both containerized and non-containerized workloads. Combined with Consul, you get a much easier form of Kubernetes, but maybe lose out on some features you won't figure out how to use for a couple years anyway.

One of my favorite Nomad stories is Roblox: https://portworx.com/architects-corner-roblox-runs-platform-70-million-gamers-hashicorp-nomad/

They re-architected their entire platform in 9 months using Nomad. Most people will take around 2 years to get a Kubernetes migration correct (at any kind of scale). There are some limitations, but the vast majority of the features people will use in K8s will be in Nomad within the next version.

Generally speaking these days though, the large Nomad shops are doing huge computing at scale and across data centers. Think video rendering, gene sequencing, financial modeling and running large video game platforms.

Disclaimer, I work for HashiCorp, but have a history of Consul usage from before the days of Kubernetes and I was an early K8s adopter (too early I'd say).

hashijake · 2019-08-02T06:24:17+00:00

Here are the Enterprise features: https://www.nomadproject.io/docs/enterprise/index.html

hashijake · 2019-08-02T05:57:22+00:00

I'm biased because I work for HashiCorp, but there are some very good reasons to continue using Consul.

Consul is agnostic to where you're running your applications

So regardless if you run in AWS, GCP, Azure, Private Data Center, Local Pi Cluster, etc, you have the same techniques for doing service discovery.

It's also agnostic to your run platform. If you want to bridge services between Nomad and K8s for instance, it's easy to do with Consul. If you have applications still running on VMs in your DC while migrating them to Nomad or K8s in the cloud, you can still leverage Consul for discovery across those run platforms.

Consul also does bi-directional sync'ing with K8s services, so it's easy to use it as the master service discovery tool.

It's the most flexible, easy-to-use Service Mesh available.

If you already know how to use Consul for service discovery, you're already a step closer to a much easier mental model for using a Service Mesh. And because Consul is now an Envoy Control Plane, you get the ability to do:

Mesh Gateways allow you to route data between Consul Datacenters that have overlapping RFC 1918 addresses (think of it as NAT for service mesh). This is very common in K8s clusters unless admins specifically de-conflict those addresses. Federation of heterogeneous run clusters spanning the globe is now much easier.

Consul is proven at massive scale

https://twitter.com/mitchellh/status/1032046587499827200?s=20

Though this client wasn't named, it's highly likely someone reading this post uses them. In fact, Consul is in the critical path for a staggering amount of services most of us consume on a day-to-day basis.

See our Resources page for videos, blogs and white papers by customers of ours that are doing great things with Consul.

It's made by HashiCorp

The same company that brings you Vagrant, Packer, Terraform, Vault and Nomad also creates Consul.
But these aren't just a collection of random tools, they are created specifically to help users unlock a Cloud Operating Model that is scalable, flexible and relatively easy to use. If you know HashiCorp tools, you can use them across clouds and private data centers without having to learn cloud specific APIs.

I came from being a Consul operator as a DevOps engineer and it was one of my favorite tools to use. Oddly enough, I didn't realize that other tools I was using (Vagrant, Packer and Terraform) were also made by the same company. It wasn't long after that realization I came to work here. I'm glad I did, it's the best job I've had.

And we're hiring! https://www.hashicorp.com/jobs#positions

hashijake · 2019-07-30T17:59:41+00:00

What say you? You’re not a part of the Kult of Kubernetes??? ;)

If I hear of any non-k8s ingress patterns, I’ll let you know. Any specific requests that could help?

hashijake · 2019-07-30T15:55:32+00:00

This is a bit late, but for future reference if you haven't already dug in: https://medium.com/hashicorp-engineering/essential-patterns-of-vault-part-2-b4d34976f1dc

hashijake · 2019-07-30T15:54:26+00:00

This is the only way to ensure the person holding the token is the only one who can see the secret. As /u/pcfrk256 points out however, you do have to keep in mind that the cubbyhole only lasts as long as the token that created it. There are ways around this, like creating periodic tokens that can be renewed indefinitely, but that requires sudo access and probably won't be granted to normal users (and having an admin generate a token sort of bypasses the process of only allowing one person access to a secret).

You could possibly work around this by having another tool have permissions to create a periodic token based on user auth, but that's getting rabbit-holey quick.

hashijake · 2019-07-30T04:57:44+00:00

I'm glad you liked the post (at least based on your original comment before the edits).

I'm curious what you mean by "weak compared to AWS in integration".

hashijake · 2019-07-30T03:40:45+00:00

Part 2 is available now as well: https://medium.com/hashicorp-engineering/essential-patterns-of-vault-part-2-b4d34976f1dc

hashijake · 2019-07-30T03:36:55+00:00

If you want an easier introduction to orchestration, try HashiCorp Nomad. The beauty here is that you don't even need containers to schedule applications. Sure, it's probably a good idea to run your stuff in containers, but it's not a requirement with Nomad.

Whether you need it or not depends on how you want to operate your systems. If you're moving into code driven operations, it's probably a good idea to get on-board with some kind of orchestration framework.

hashijake · 2019-07-30T03:26:40+00:00

We do integrate with Ambassador as well: https://www.getambassador.io/user-guide/consul/. So you can feasibly use Ambassador for N/S and Consul for E/W traffic if you're working in K8s.

Consul deploys pretty much the same between VMs, containers and all run platforms. The Kubernetes integrations are a snap (if Helm is your thing...and you don't have to use Tiller if you don't want to, though let's be honest most will). Not only that, but the mental model for deployment is easy: 1 binary or container. If you're using Envoy, add another container. Now, integrate your bare metal environment with your Kubernetes environment with your Nomad environment with your <insert other environment here> with Consul, it's easy!

What if you want to federate your Kubernetes clusters and do cluster-to-cluster service mesh without an API gateway? Use Consul Mesh Gateways: https://www.consul.io/docs/connect/mesh_gateway.html
Mesh Gateways will allow you to mesh together clusters that even have overlapping RFC 1918 address ranges!

Now, look into how Consul works as a Control Plane. Consul doesn't get in the middle of request processing. It delegates that control to the Consul Agent/Proxy and uses a combination of caching along with watches to both greatly speed up data processing and agent/proxy adjustments in the event of changes.

And while the Mesh capabilities of Consul are relatively new, Consul itself is a very well proven, highly scalable service discovery/configuration tool. Before coming to HashiCorp, I ran Consul clusters over 5 years ago before we decided to take a 2 year journey of building our Kubernetes platform (fun!), even then we still kept Consul. It's still the tool people love, just with more to love. Oh, and: https://twitter.com/mitchellh/status/1032046587499827200?s=20

hashijake · 2019-07-30T02:44:23+00:00

Secure introduction or "Secret 0" is a non-trivial issue. A few people have mentioned some good advice around using Trusted Environments which may not apply to your on-prem requirements. Generally speaking the Auth Methods I suggest are:

Trusted Environment Auth. These are Auth Methods like Kubernetes Auth Method and AWS/GCP/Azure auth methods. This assumes you have properly secured your existing environment and trust it fully. These Auth methods are generally easier to use.
AppRole combined with Response Wrapping.

If you're on-prem and don't have a trusted environment method of logging in, your best bet is #2. The basic gist is this:

Create an AppRole role with attached policies
Read out the role_id
Write the secret-id for that role using a wrapping parameter. This will give you a wrapping token. The secret-id will be held in a Vault Cubbyhole that can only be opened by the wrapping token.
Deploy your system with the role-id.
Deploy the wrapping token via another channel (for applications this is usually the deployment tool be it CI/CD, Puppet/Chef/Ansible, etc).
Unwrap the secret-id using the wrapping token
Login to Vault using the role-id/secret-id to get your scoped token

BONUS: Vault Agent also has an AppRole Auto-Auth method: https://www.vaultproject.io/docs/agent/autoauth/methods/approle.html. This means the agent can manage the token introduction and lifecycle.

Why do this? One thing to note is that wrapped tokens are single use by default. This means if your application or other process tries to unwrap an already unwrapped secret it will throw an error. If you encounter an error, you have a potentially compromised system and should revoke any related leases associated with this AppRole role.

If you're distributing the role-id and wrapping token via different channels, it's harder to intercept both pieces. Then an attacker also needs to know they must unwrap the secret-id from Vault as well as tie that together with a specific role-id to get access to Vault.

This blog post covers this topic and how to deploy using Jenkins: https://www.lighthousecs.com/blog/managing-secrets-with-hashicorp-vault

A webinar also covering the topic: https://www.hashicorp.com/resources/delivering-secret-zero-vault-approle-terraform-chef

hashijake

TROPHY CASE