Styjun

How to respectfully refuse to assist co-workers with IT issues?

Why did Khan ask Admiral James T. Kirk about Project Genesis?

Are modern clipless shoes and pedals that much better than toe clips and straps?

Is there any method of inflicting the incapacitated condition and no other condition?

The Knight's estate

Why doesn't 'd /= d' throw a division by zero exception?

Dataset human solutions for OR problems

What to say to a student who has failed?

A discontinuity in an integral

Thank God it's Friday, tomorrow is THE weekend. Why the definite article?

Justifying the use of directed energy weapons

How should I face my manager if I make a mistake because a senior coworker explained something incorrectly to me?

Command in bash shell script to find path to that script?

How do you harvest carrots in creative mode?

Would the Republic of Ireland and Northern Ireland be interested in reuniting?

Can a Rogue PC teach an NPC to perform Sneak Attack?

Is using a hyperlink to close a modal a poor design decision?

Would it be possible to have a GMO that produces chocolate?

Are the players on the same team as the DM?

antonym of "billable"

Was there ever a treaty between 2 entities with significantly different translations to the detriment of one party?

Examples of topos that are not ordinary spaces

How to prevent clipped screen edges on my TV, HDMI-connected?

Is immersion of utensils (tevila) valid before koshering (hagala)?

Path of data on Google Cloud Platform for NiFi

What is the difference between Google App Engine and Google Compute Engine?Google Cloud gsutil instructions say update but there is no update commandAccessing files in the Google Cloud Storage from two different google cloud projectsHow to split the my flowfile content based on 'n'?RSA Digital signature verification using public key in apache nifiNiFI “unable to find flowfile content”Nifi and Avro: Convert the data and metadata into avro file using specified avro schema?Dataflow from Azure to Google Cloud Platform using NiFihow to connect to external ip of google cloud platform from local browser

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

0

I want to give location of input folder in GetFile processor in NiFi. The path which I provided is incorrect according to NiFi. It still shows the warning sign.
I followed this link but it couldn't provide much help. Below is my path which I used in input path property in GetFile processor:

/browser/<bucket-name>/

I even tried below links

gs://<bucket-name>

or

gs://browser/<bucket-name>

None of them helped.

As suggested by Shu, I created one ListGCSBucket processor and created a new GCPCredentialsService for it but when I ran this processor it gives an error :
com.google.cloud.storage.StorageException: Not Found for bucket. Below are my few config:
Processor Details:

GCSControllerService Details:

In the last image, it is not allowing me to change the values to true. So here are my complete details. Please guide where I am going wrong. Any help is much appreciated.

hadoop google-cloud-platform bigdata apache-nifi

share|improve this question

edited Mar 28 at 7:09

Siddhesh Kalgaonkar

asked Mar 27 at 17:38

Siddhesh KalgaonkarSiddhesh Kalgaonkar

42844 silver badges2525 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Could you please add warning logs to the question?
  
  – dhnsh
  Mar 27 at 18:07
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  It shows invalid path exception
  
  – Siddhesh Kalgaonkar
  Mar 27 at 18:16
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Do u have the path /browser/<bucket-name>/ that specified in GetFile, on all NiFi installed nodes?
  
  – dhnsh
  Mar 27 at 18:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  My data is in bucket on cloud. So what would be the correct URL in this case for data which is under particular bucket
  
  – Siddhesh Kalgaonkar
  Mar 27 at 18:28
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks for clarification, i have added the answer to the question, please check the answer and let us know if there are any issues..!
  
  – dhnsh
  Mar 27 at 18:38
  

  
  
  
  

add a comment | 

0

I want to give location of input folder in GetFile processor in NiFi. The path which I provided is incorrect according to NiFi. It still shows the warning sign.
I followed this link but it couldn't provide much help. Below is my path which I used in input path property in GetFile processor:

/browser/<bucket-name>/

I even tried below links

gs://<bucket-name>

or

gs://browser/<bucket-name>

None of them helped.

As suggested by Shu, I created one ListGCSBucket processor and created a new GCPCredentialsService for it but when I ran this processor it gives an error :
com.google.cloud.storage.StorageException: Not Found for bucket. Below are my few config:
Processor Details:

GCSControllerService Details:

In the last image, it is not allowing me to change the values to true. So here are my complete details. Please guide where I am going wrong. Any help is much appreciated.

hadoop google-cloud-platform bigdata apache-nifi

share|improve this question

edited Mar 28 at 7:09

Siddhesh Kalgaonkar

asked Mar 27 at 17:38

Siddhesh KalgaonkarSiddhesh Kalgaonkar

42844 silver badges2525 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Could you please add warning logs to the question?
  
  – dhnsh
  Mar 27 at 18:07
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  It shows invalid path exception
  
  – Siddhesh Kalgaonkar
  Mar 27 at 18:16
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Do u have the path /browser/<bucket-name>/ that specified in GetFile, on all NiFi installed nodes?
  
  – dhnsh
  Mar 27 at 18:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  My data is in bucket on cloud. So what would be the correct URL in this case for data which is under particular bucket
  
  – Siddhesh Kalgaonkar
  Mar 27 at 18:28
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks for clarification, i have added the answer to the question, please check the answer and let us know if there are any issues..!
  
  – dhnsh
  Mar 27 at 18:38
  

  
  
  
  

add a comment | 

I want to give location of input folder in GetFile processor in NiFi. The path which I provided is incorrect according to NiFi. It still shows the warning sign.
I followed this link but it couldn't provide much help. Below is my path which I used in input path property in GetFile processor:

/browser/<bucket-name>/

I even tried below links

gs://<bucket-name>

or

gs://browser/<bucket-name>

None of them helped.

As suggested by Shu, I created one ListGCSBucket processor and created a new GCPCredentialsService for it but when I ran this processor it gives an error :
com.google.cloud.storage.StorageException: Not Found for bucket. Below are my few config:
Processor Details:

GCSControllerService Details:

In the last image, it is not allowing me to change the values to true. So here are my complete details. Please guide where I am going wrong. Any help is much appreciated.

hadoop google-cloud-platform bigdata apache-nifi

share|improve this question

edited Mar 28 at 7:09

Siddhesh Kalgaonkar

asked Mar 27 at 17:38

Siddhesh KalgaonkarSiddhesh Kalgaonkar

42844 silver badges2525 bronze badges

/browser/<bucket-name>/

gs://<bucket-name>

gs://browser/<bucket-name>

share|improve this question

edited Mar 28 at 7:09

Siddhesh Kalgaonkar

asked Mar 27 at 17:38

Siddhesh KalgaonkarSiddhesh Kalgaonkar

42844 silver badges2525 bronze badges

share|improve this question

edited Mar 28 at 7:09

Siddhesh Kalgaonkar

asked Mar 27 at 17:38

Siddhesh KalgaonkarSiddhesh Kalgaonkar

42844 silver badges2525 bronze badges

asked Mar 27 at 17:38

Siddhesh KalgaonkarSiddhesh Kalgaonkar

42844 silver badges2525 bronze badges

asked Mar 27 at 17:38

Siddhesh KalgaonkarSiddhesh Kalgaonkar

42844 silver badges2525 bronze badges

1 Answer
1

active

oldest

votes

2

As per the comments on the question.

GetFile processor looks for input path that specified on all NiFi installed nodes.

Use ListGCSBucket + FetchGCSObject processors to get the data from GoogleCloudPlatform.

For more references refer this and this links to use list+fetch processors in nifi.

share|improve this answer

answered Mar 27 at 18:37

dhnshdhnsh

7,36222 gold badges77 silver badges2121 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I have edited my question with screenshots. could you please help me out here ? Thanks
  
  – Siddhesh Kalgaonkar
  Mar 28 at 7:09
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiddheshKalgaonkar, disable the GCPCredentialsControlerService and then you are able to change the value to true. there is a post in nifi dev list regards to similar error: apache-nifi.1125220.n5.nabble.com/…
  
  – dhnsh
  Mar 28 at 14:22
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Yeah I changed the value to true and I entered the right bucket path i guess but still it doesn't read the data when I start the processor. Can you provide me some sample
  
  – Siddhesh Kalgaonkar
  Mar 28 at 14:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Any updates on this please ? can anyone help me here ?
  
  – Siddhesh Kalgaonkar
  Apr 1 at 14:17
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiddheshKalgaonkar, it seems to be we need to just give bucket name in bucket property value of ListGCSBucket processor instead of full url. some reference that will help: community.hortonworks.com/questions/115115/… medium.com/datafabric/…
  
  – dhnsh
  Apr 1 at 14:54
  
  

  
  
  
  

 | 
show 2 more comments

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55383422%2fpath-of-data-on-google-cloud-platform-for-nifi%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

2

As per the comments on the question.

GetFile processor looks for input path that specified on all NiFi installed nodes.

Use ListGCSBucket + FetchGCSObject processors to get the data from GoogleCloudPlatform.

For more references refer this and this links to use list+fetch processors in nifi.

share|improve this answer

answered Mar 27 at 18:37

dhnshdhnsh

7,36222 gold badges77 silver badges2121 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I have edited my question with screenshots. could you please help me out here ? Thanks
  
  – Siddhesh Kalgaonkar
  Mar 28 at 7:09
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiddheshKalgaonkar, disable the GCPCredentialsControlerService and then you are able to change the value to true. there is a post in nifi dev list regards to similar error: apache-nifi.1125220.n5.nabble.com/…
  
  – dhnsh
  Mar 28 at 14:22
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Yeah I changed the value to true and I entered the right bucket path i guess but still it doesn't read the data when I start the processor. Can you provide me some sample
  
  – Siddhesh Kalgaonkar
  Mar 28 at 14:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Any updates on this please ? can anyone help me here ?
  
  – Siddhesh Kalgaonkar
  Apr 1 at 14:17
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiddheshKalgaonkar, it seems to be we need to just give bucket name in bucket property value of ListGCSBucket processor instead of full url. some reference that will help: community.hortonworks.com/questions/115115/… medium.com/datafabric/…
  
  – dhnsh
  Apr 1 at 14:54
  
  

  
  
  
  

 | 
show 2 more comments

2

As per the comments on the question.

GetFile processor looks for input path that specified on all NiFi installed nodes.

Use ListGCSBucket + FetchGCSObject processors to get the data from GoogleCloudPlatform.

For more references refer this and this links to use list+fetch processors in nifi.

share|improve this answer

answered Mar 27 at 18:37

dhnshdhnsh

7,36222 gold badges77 silver badges2121 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I have edited my question with screenshots. could you please help me out here ? Thanks
  
  – Siddhesh Kalgaonkar
  Mar 28 at 7:09
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiddheshKalgaonkar, disable the GCPCredentialsControlerService and then you are able to change the value to true. there is a post in nifi dev list regards to similar error: apache-nifi.1125220.n5.nabble.com/…
  
  – dhnsh
  Mar 28 at 14:22
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Yeah I changed the value to true and I entered the right bucket path i guess but still it doesn't read the data when I start the processor. Can you provide me some sample
  
  – Siddhesh Kalgaonkar
  Mar 28 at 14:27
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Any updates on this please ? can anyone help me here ?
  
  – Siddhesh Kalgaonkar
  Apr 1 at 14:17
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @SiddheshKalgaonkar, it seems to be we need to just give bucket name in bucket property value of ListGCSBucket processor instead of full url. some reference that will help: community.hortonworks.com/questions/115115/… medium.com/datafabric/…
  
  – dhnsh
  Apr 1 at 14:54
  
  

  
  
  
  

 | 
show 2 more comments

As per the comments on the question.

GetFile processor looks for input path that specified on all NiFi installed nodes.

Use ListGCSBucket + FetchGCSObject processors to get the data from GoogleCloudPlatform.

For more references refer this and this links to use list+fetch processors in nifi.

share|improve this answer

answered Mar 27 at 18:37

dhnshdhnsh

7,36222 gold badges77 silver badges2121 bronze badges

share|improve this answer

answered Mar 27 at 18:37

dhnshdhnsh

7,36222 gold badges77 silver badges2121 bronze badges

answered Mar 27 at 18:37

dhnshdhnsh

7,36222 gold badges77 silver badges2121 bronze badges

answered Mar 27 at 18:37

dhnshdhnsh

7,36222 gold badges77 silver badges2121 bronze badges