Using custom .env path only works from workspace root

[stevebeauge]

(moved from NiklasPor/nx-remotecache-azure#22)

Hello,

I don't know if I mis understood something, but I'm having troubles to configure the custom cache (using azure in my case).

When I run pnpm nx run "@myorg/myproject:build" --verbose from the root of the workspace, I can see cache populated.

But if I run cd packages/myproject then pnpm nx build --verbose, the cache does not works.

I get this error:

Warning: Failed to store cache to Azure Blob Storage
File: 12345658363156603647.tar.gz
Error: Did not pass valid container. Supply the container either via env or nx.json.

To setup the cache, I configured my root nx.json file like this:

{
	"tasksRunnerOptions": {
		"default": {
			"runner": "nx-remotecache-azure",
			"options": {
				"maxParallel": 4,
				"dotenvPath": ".env.secrets",
				"cacheableOperations": ["build", "test", "lint", "e2e", "release"]
			}
		}
	},
     ....
}

And I created a file .env.secrets sibling to the nx.json file which contains:

# Chaine de connexion du blob storage stockant le cache de Nx
NXCACHE_AZURE_SAS_URL=https://mystorageaccount.blob.core.windows.net/nx-cache?sp=racwdli&st=2024-03-26T07:59:15Z&se=2028-03-26T14:59:15Z&spr=https&sv=2022-11-02&sr=c&sig=abcabcabcabcabc%3D

The full error is:

------------------------------------------------------------------------------
Warning: Failed to store cache to Azure Blob Storage
File: 12482558363156603647.tar.gz
Error: Did not pass valid container. Supply the container either via env or nx.json.
------------------------------------------------------------------------------
Error: Did not pass valid container. Supply the container either via env or nx.json.
    at getBlockBlobClient (C:\path\to\node_modules\.pnpm\nx-remotecache-azure@18.0.0_nx@18.0.8\node_modules\nx-remotecache-azure\index.js:26:15)
    at blob (C:\path\to\node_modules\.pnpm\nx-remotecache-azure@18.0.0_nx@18.0.8\node_modules\nx-remotecache-azure\index.js:49:32)
    at storeFile (C:\path\to\node_modules\.pnpm\nx-remotecache-azure@18.0.0_nx@18.0.8\node_modules\nx-remotecache-azure\index.js:54:42)
    at C:\path\to\node_modules\.pnpm\nx-remotecache-custom@18.0.0_nx@18.0.8\node_modules\nx-remotecache-custom\get-safe-remote-cache-implementation.js:30:30
    at Object.store (C:\path\to\node_modules\.pnpm\nx-remotecache-custom@18.0.0_nx@18.0.8\node_modules\nx-remotecache-custom\create-remote-cache-store.js:17:11)
    at async C:\path\to\node_modules\.pnpm\nx@18.0.8\node_modules\nx\src\tasks-runner\cache.js:99:17
    at async _try (C:\path\to\node_modules\.pnpm\nx@18.0.8\node_modules\nx\src\tasks-runner\cache.js:222:24)
    at async Promise.all (index 0)
    at async TaskOrchestrator.postRunSteps (C:\path\to\node_modules\.pnpm\nx@18.0.8\node_modules\nx\src\tasks-runner\task-orchestrator.js:261:13)
    at async TaskOrchestrator.applyFromCacheOrRunTask (C:\path\to\node_modules\.pnpm\nx@18.0.8\node_modules\nx\src\tasks-runner\task-orchestrator.js:216:9)

When running from the workspace root, I get, in the output

Stored output to remote cache: Azure Blob Storage
File: 14400678733458654225.tar.gz

If I set up manually the env variable using

$env:NXCACHE_AZURE_SAS_URL="https://mystorageaccount.blob.core.windows.net/nx-cache?sp=racwdli&st=2024-03-26T07:59:15Z&se=2028-03-26T14:59:15Z&spr=https&sv=2022-11-02&sr=c&sig=abcabcabcabcabc%3D"

The cache always works, so I guess there's something wrong with the way the local .env.secrets file is read.

I tried to explicitely targets the env file at workspace root using "dotenvPath": "{workspaceroot}/.env.secrets", but there's no more success.

If it matters, pnpm why nx* give:

devDependencies:
nx 18.0.8
└─┬ @nrwl/tao 18.0.8
  └── nx 18.0.8
nx-remotecache-azure 18.0.0
├─┬ nx 18.0.8 peer
│ └─┬ @nrwl/tao 18.0.8
│   └── nx 18.0.8
└─┬ nx-remotecache-custom 18.0.0
  └─┬ nx 18.0.8 peer
    └─┬ @nrwl/tao 18.0.8
      └── nx 18.0.8

And my node version is 16.20.2

Thanks

[stevebeauge]

I think the issue come from there:

https://github.com/NiklasPor/nx-remotecache-custom/blob/fb48a25285fee19589f4ddd9b2a7f235bc2dd40b/lib/init-env.ts#L7C1-L10C4

I believe the dotenv path provided is relative the current location where the nx command is ran.

I quickly prototyped a workaround:

import dotenv from "dotenv";
import { workspaceRoot } from "nx/src/utils/workspace-root";
import { CustomRunnerOptions } from "./types/custom-runner-options";
/**
 * Initializes the environment variables.
 */
export const initEnv = (options: CustomRunnerOptions) => {
  if (options.dotenv !== false) {
    const dotenvConfig = {
      path: options.dotenvPath?.replace("{workspaceRoot}", workspaceRoot),
    };
    console.log("🍕 dotenvConfig", dotenvConfig);
    dotenv.config(dotenvConfig);
  }
};

This way, the token {workspaceRoot} is properly resolved if present.

I can submit a PR with it if you think this is the solution, but I'm note sure if it's the proper way to handle it, and there's other token to take into account.

[stevebeauge]

Seems to be fixed in the latest version. Not sure if it was a nx or this lib issue actually.

[stevebeauge]

Nevermind. The issue is still present (had the env files loaded elsewhere). sorry