allow variable in provider field

[dmrzzz]

I'm using multiple AWS providers to deploy resources into different regions, but I want my modules to be generic with respect to which regions.

Ideally I would write a module like this:

variable provider { default = "aws" }

resource "aws_vpc" "vpc" {
    # works:
    #provider = "aws.us-east-1"
    # does not work:
    provider = "${var.provider}"
    cidr_block = "192.168.0.0/16"
    tags { Name = "TEST - DELETE ME" }
}

and invoke it like this:

provider "aws" {
  region = "us-east-2"
}

provider "aws" {
  alias = "us-east-1"
  region = "us-east-1"
}

# do other stuff using the default AWS provider

# tell this module to use aws.us-east-1
module "module1" {
  source = "./testmodule"
  provider = "aws.us-east-1"
}

but that fails on terraform get:

dmrz@golbez:~/tmp/tftest2$ terraform get
Get: file:///home/dmrz/tmp/tftest2/testmodule
Error loading Terraform: 1 error(s) occurred:

* module module1: provider alias must be defined by the module or a parent: ${var.provider}

It works fine if I instead hardcode the provider field value within the module, but then my module isn't generic.

Note: the resource I really want to use this for at this moment is aws_cloudwatch_metric_alarm, but aws_vpc behaves the same way and makes a much simpler test case.

Thanks for all the great work on Terraform!

[kevintsai]

I also ran into this issue. Hope it can be improved. Thanks.

[apparentlymart]

This is somewhat related to #1819.

[kkarimi]

This is quite necessary in order to work effectively with modules with a multi region AWS setup

[ericfode]

This is causing sadness for us.

[nyetsche]

Also causing sadness for us :(

Here's the example of what I'm hoping to accomplish:

resource "aws_kms_key" "cluster_cmk" {
  count       = "${var.cluster_number}"
  provider = "aws.${element(values(var.regionmap), count.index)}"
}

provider "aws" {
  alias = "us-east-1"
  region = "us-east-1"
}

regionmap = {
  "clusternickname" = "us-east-1"
}

In other words, have multiple KMS keys, one per cluster, and each cluster could be in a different region. So do a map lookup.

Fails with this:

* module root: 1 error(s) occurred:

* module : provider alias must be defined by the module or a parent: aws.${element(values(var.regionmap), count.index)}

I poked around the code but couldn't figure out where a resource overrides the default provider (where the fix should go). If someone has hints, it's important enough I'd like to hack on a PR

[demonwy]

+1

[gillingw]

This is an issue for us too having to duplicate modules for each region.

[apparentlymart]

Hi all,

I'm sorry there hasn't been any movement here. This is a tricky issue to resolve because we need to understand provider usage very early on in the Terraform run, before we have enough context to do interpolations. So far a way to implement this as requested has eluded us.

A different take on the problem might be simpler to implement. I've not thought this through all the way yet so I can't promise this is how this will eventually end up, but just wanted to capture this here for future reference and feedback:

module "foo" {
  source = "./foo"

  # ...module arguments, as usual...

  providers {
    aws = "aws.use1"
  }
}

This hypothetical (not yet working!) providers block could allow customizing how providers inherit so that a provider with an alias in the parent module can be unaliased in the child module. By handing this in a first-class way we'd avoid the need to support arbitrary interpolations for it and thus be able to resolve it early in configuration processing.

I understand that this alternative approach is still not as general as the interpolation-based solution requested, but I think it would be adequate to meet the use-case of giving a child module access to an aliased provider without the child module itself needing to know about the alias.

[dmrzzz]

@apparentlymart, your idea above would solve my immediate problem.

That said, I keep thinking about how nice it would be to have some sort of limited interpolation support available earlier in the run, such that many parameters which currently allow no interpolation at all could instead allow limited interpolation (almost like a preprocessor macro replacement step, but using the same familiar interpolation syntax).

Obviously we wouldn't be able to interpolate anything at this early stage whose value depends on another resource or data source, but for my OP example code the value in question is knowable as soon as we parse the files, because it's already hardcoded elsewhere in those same files. Being able to interpolate such locally-known variable values early on would, from my perspective, cleanly solve not only this issue but also e.g. #13022 (and maybe some others too) without having to introduce new first-class syntax.

[jbarreneche]

Hi! The overriding provisioner in the module would work for our use case and would be more clean that what we are currently doing.

Just for the record, for the people who need to handle multiple regions or multiple aws accounts this works:

In the module

provider "aws" {
  alias = "module"
  region = "${var.region}" # Or some way to infer the region from the module variables
  # This is in case you want to use multiple accounts
  assume_role {
    role_arn = "arn:aws:iam::${var.account)}:role/terraform"  # Or some way to infer the account number from the module variables, we use a map for naming the accounts.
  }
}
resource "aws_..." "foo" {
  provider = "aws.module"
  ...
}

The assume_role is optional in case you use multiple accounts (and needs all accounts to have a terraform role).
Sadly you can't forget to put the provider (which is a common mistake), and you can't override the "default" or use another that's in use (As far as we tested it). But it allows you to use a module in the same tfstate with multiple accounts/regions.

module "foo" {
  account = "12312312"
  region = "us-east-1"
}

module "bar" {
  account = "32132131"
  region = "sa-east-1"
}

[gillingw]

Thank you @jbarreneche this is working for us.

[paulcollinsiii]

So @jbarreneche first up thank you for that work around. For creating resources it works perfectly.
The downside to it comes when I'm trying to tear down a module that was created with this. At least with version 0.9.5 when I tried to remove a module that included a

provider "aws" {
  region = "${var.aws_region}"
  alias = "aws.module"
}

Then when starting a plan that would remove that module terraform would ask what region it applied to and then barf during the apply. If I went about it doing a
terraform plan -out=tf.plan -target module.mymodule -destroy
applied that, and THEN removed the module "mymodule { ... } line it was okay.

Anyhow, just a note for future me when I run into this gotcha again.

[joshmyers]

We have a set of "DR regions", which could be one or more. Not being able to have interpolation in the provider makes this not possible to use:

resource "aws_db_event_subscription" "read_replica_in_another_region" {
  count     = "${length(split(" ",var.dr_regions))}"
  provider = "aws.${element(split(" ",var.dr_regions),count.index)}"

This is not using a module but I think the workaround above would not work in my scenario. Any other ideas?

[thomasbiddle]

This is the second or third time I've had to stop on a project I was working on due to this limitation; definitely should get some extra attention :-)

My use-case: AWS Config

I have a set of rules that I want to apply to every region in AWS. I'm only using a couple, but I want to have config running on all of my regions in-case something gets added where we don't normally work without my knowing. I'd love to define all my rules in a module, and just attach it to every region!

[cfullerton]

This is a problem for us too

[apparentlymart]

Hi all!

An approach like I sketched in my earlier comment was implemented in Terraform 0.11, allowing generic modules to receive aliased providers from the parent module.

I believe this addresses the use-case that motivated this proposal. Sorry I forgot to update this while we were announcing the 0.11 release.

Since the stated problem is now solved, I'm going to close this. If others in future find use-cases that aren't addressed I'd ask that they open a fresh issue so we can discuss each use-case separately.

Thanks for sharing your use-cases, everyone!

[tamsky]

For reference: #16379 adds "module provider inheritance"

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.