insert host name to ip addresses #222
Comments
Alisher-Nabiev
changed the title
|
Hello @Alisher-Nabiev
Please outline the issue in as much detail as possible. What is working, what is not, what are the expected vs. current results, etc. |
|
ok, so this is our table: this is filter config: and we don't see host_name in the index. 0|pastash-logs | [Wed, 03 Jul 2024 10:37:51 GMT] INFO test query! SELECT host_name from ip_list WHERE ip_number = (?) limit 1; 8883 |
|
The following test succeeds for me. I created a dummy sqlite db Then installed the sqlite plugin And execute the following test recipe: I get the expected results: |
|
Hi, thank you for your response. i tried to create a new database with the same configuration as your example. I just substituted my hosts. but nothing on elastic side no new field in the indecies. log: /root/.pm2/logs/pastash-logs-error.log last 15 lines: |
|
Have you installed the same version? I do not see any useful output. You might want to add an |
|
hi, yes we install the new version. i will share with you the stdout{} in privet |
|
As you can see above my example works and its served with a fully reproducible case and each step used. You should rather find out why your process is exiting with errors in loop
|
|
As you can see, the error occurred yesterday, possibly because I restarted the PM2 when making new changes in the configuration file. Also, we receive data constantly in the Elastic, and we don't observe any unexpected exits from the PM2 process. |
|
I'm sorry but the issue is on your side and our example appears to be working - you need to investigate your setup, as without more information, I'm afraid there's nothing we can do. Are you even sure the pastash process has permissions to read your .db files? |
|
yes i gave to the db -rwx permission and still same issue |
|
Is there any reason why you did not attempt running our provided "demo" example? It might reveal the issue. |
|
what do you mean by your example? i created the same DB as you are but only with my IPs. and i put the same filter on the config file. if i am running the query on the instance it's retrieving the correct hostname. but on the elastic side, i can't see it. If you want me to test it in a different way, please let me know. |
|
Our example is stand alone. It uses user input string (IP) to validate the pipeline. It has no elastic or other transformations. This is why in case of issues its useful to troubleshoot a single component as opposed to the full configuration. Try it and confirm it works before moving ahead with integration in your workflow. |
|
did you reacive my privet emails with my configuration? second; [TAILING] Tailing last 15 lines for [2] process (change the value with --lines option) how did you pass the IP number to process? with input? |
|
Do not use pm2 for this test. Just run |
|
root@ip:/home/ubuntu# pastash --config_file=dumy_conf.conf |
|
As you can see it works. Perhaps try to use a different target header and/or experiment with the order of filters |
|
in the filter "target_field" is for pass the field to the output and from there move it to elastic? as you can see here: ip_proto { target field is creating "service" field and passing this to elastic: 0|pastash-logs | [STDOUT] { but from here not: |
|
I see the issue. You don't have alignment between the select (host) and the target_field. |
|
great!! it works thanks |
we are attempting to use an SQLite filter to capture incoming IP addresses in the index and then add a name to each IP address from the database.
the IPs are custom and not related to DNS. we want to take IPs and attach our client's hostnames.
the list in the DB will be updated frequently.
is there any way to achieve that (reverse DNS is not fit for this task)
sqlite {
db => '/home/folder/userdata_qa.db'
query => 'SELECT service from ip_list WHERE ip_number=(?) limit 1;'
source_field => 'ip_v4'
target_field => 'host_name'
}
this is not working well for us.
The text was updated successfully, but these errors were encountered: