Demonstrate how to index metadata into a Elasticsearch database (AWS Service) and search on it. This sample recursively iterate through all projects of the selected BIM 360 Account (Hub), then into the Project Files folder, then subfolders, then the last version of an item, get all the metadata and send to Elasticsearch as a text (not JSON). When a new file is uploaded, a webhook triggers and update the index. Once the index is done, one can search and redirect to the BIM 360 file.
- Forge Account: Learn how to create a Forge Account, activate subscription and create an app at this tutorial.
- Visual Studio: Either Community (Windows) or Code (Windows, MacOS).
- .NET Core basic knowledge with C#
- ngrok: Routing tool, download here
- MongoDB: noSQL database, learn more. Or use a online version via mLab (this is used on this sample)
- AWS Account with Elasticsearch: create or use your AWS Account, learn more.
Clone this project or download it. It's recommended to install GitHub desktop. To clone it via command line, use the following (Terminal on MacOSX/Linux, Git Shell on Windows):
git clone https://github.com/autodesk-forge/forge-bim360search
Amazon Elasticsearch
Create or use your AWS Account. Go to Elasticseach dashboard and Create new domain.
- Step 1: you may consider using
Development and testing:Development and testing. - Step 2: type your
Elasticsearch domain name. Other settings as default or adjust as needed. - Step 3: depending on your
Network configuration, or selectPublic access. Select the appropriateAccess policy, this sample uses Allow or deny access to one or more IAM users. Select the appropriate user. This step requires a IAM user. - Step 4: review and confirm.
The creating may take a few minutes, after that take note of the Endpoint.
MongoDB
MongoDB is a no-SQL database based on "documents", which stores JSON-like data. For testing purpouses, you can either use local or live. For cloud environment, try MongoDB Atlas (offers a free tier). With MongoDB Atlas you can set up an account for free and create clustered instances, intructions:
- Create a account on MongoDB Atlas.
- Under "Collections", create a new database (e.g. named
bim360search) with a collection (e.g. namedusers). - Under "Command Line Tools", whitelist the IP address to access the database, see this tutorial. If the sample is running on Heroku, you'll need to open to all (IP
0.0.0.0/0). Create a new user to access the database.
At this point the connection string should be in the form of mongodb+srv://<username>:<password>@clusterX-a1b2c4.mongodb.net/inventor2revit?retryWrites=true. Learn more here
There are several tools to view your database, Robo 3T (formerly Robomongo) is a free lightweight GUI that can be used. When it opens, follow instructions here to connect to MongoDB Atlas.
Visual Studio (Windows):
Right-click on the project, then go to Debug. Adjust the settings as shown below. For environment variable, define the following:
- ASPNETCORE_ENVIRONMENT:
Development - FORGE_CLIENT_ID:
your id here - FORGE_CLIENT_SECRET:
your secret here - FORGE_CALLBACK_URL:
http://localhost:3000/api/forge/callback/oauth - FORGE_WEBHOOK_URL:
http://YOUR HOST OR NGROK ADDRESS - FORGE_PARALLEL_JOBS:
Number of parallels jobs to get metadata, this affects the memory usage - OAUTH_DATABASE:
mongodb+srv://<username>:<password>@clusterX-a1b2c4.mongodb.net/bim360search?retryWrites=true - ELASTIC_SEARCH_SERVER:
https://yourSearchEndpoint.region.es.amazonaws.com - AWS_REGION:
where the Elastic Search is running, e.g. us-east-1 - AWS_ACCESS_KEY:
your AWS access key here - AWS_SECRET_KEY:
your AWS secret key here
Visual Sutdio Code (Windows, MacOS):
Open the folder, at the bottom-right, select Yes and Restore. This restores the packages (e.g. Autodesk.Forge) and creates the launch.json file. See Tips & Tricks for .NET Core on MacOS.
At the .vscode\launch.json, find the env vars and add your Forge Client ID, Secret and callback URL. Also define the ASPNETCORE_URLS variable. The end result should be as shown below:
"env": {
"ASPNETCORE_ENVIRONMENT": "Development",
"ASPNETCORE_URLS": "http://localhost:3000",
"FORGE_CLIENT_ID": "your id here",
"FORGE_CLIENT_SECRET": "your secret here",
"FORGE_CALLBACK_URL": "http://localhost:3000/api/forge/callback/oauth",
"FORGE_WEBHOOK_URL": "http://YOUR HOST OR NGROK ADDRESS",
"FORGE_PARALLEL_JOBS": "20",
"OAUTH_DATABASE": "mongodb+srv://<username>:<password>@clusterX-a1b2c4.mongodb.net/bim360search?retryWrites=true",
"ELASTIC_SEARCH_SERVER": "https://search-yourElasticSearch.region.es.amazonaws.com",
"AWS_REGION": "where the Elastic Search is running, e.g. us-east-1",
"AWS_ACCESS_KEY": "your AWS access key here",
"AWS_SECRET_KEY": "your AWS secret key here"
},Open http://localhost:3000 to start the app. Select Index my BIM 360 Account before using (this process may take a while). Check the http://localhost:3000/dashboard to see the jobs running (Hangfire dashboard).
Due to memory usage, it's likely that this sample will crash on Heroku free & hobby tier (limited to 512Mb of memory). It may work if drawing do not have much metadata.
To deploy this application to Heroku, the Callback URL for Forge must use your .herokuapp.com address. After clicking on the button below, at the Heroku Create New App page, set your Client ID, Secret and Callback URL for Forge.
Documentation:
Other APIs:
- Hangfire queueing library for .NET
- MongoDB for C# driver
- Mongo Altas Database-as-a-Service for MongoDB
This sample uses .NET Core and works fine on both Windows and MacOS, see this tutorial for MacOS.
-
Cannot see my BIM 360 projects: Make sure to provision the Forge App Client ID within the BIM 360 Account, learn more here. This requires the Account Admin permission.
-
error setting certificate verify locations error: may happen on Windows, use the following:
git config --global http.sslverify "false"
This sample is licensed under the terms of the MIT License. Please see the LICENSE file for full details.
Augusto Goncalves @augustomaia, Forge Partner Development