Styjun

Does this Wild Magic result affect the sorcerer or just other creatures?

Would it be a copyright violation if I made a character’s full name refer to a song?

Old sci-fi story: radiation mutated the animals, a boy loses a limb, but it's okay because "humans used to do great with only two arms"

How does a pilot select the correct ILS when the airport has parallel runways?

Why do all the teams that I have worked with always finish a sprint without completion of all the stories?

Did the CIA blow up a Siberian pipeline in 1982?

Why use cross notes in sheet music for hip hop tracks?

Why do some professors with PhDs leave their professorships to teach high school?

Output of "$OSTYPE:6" on old releases of Mac OS X

What size of powerbank will I need to power a phone and DSLR for 2 weeks?

When to remove insignificant variables?

Really Old Stock Valuation

Can White Castle?

Why does the Saturn V have standalone inter-stage rings?

Unusual mail headers, evidence of an attempted attack. Have I been pwned?

How do I set an alias to a terminal line?

Dates on degrees don’t make sense – will people care?

Is there a maximum distance from a planet that a moon can orbit?

What did River say when she woke from her proto-comatose state?

Why do even high-end cameras often still include normal (non-cross-type) AF sensors?

Trainee keeps missing deadlines for independent learning

JSON selector class in Python

What can I do with a research project that is my university’s intellectual property?

What reason would an alien civilization have for building a Dyson Sphere (or Swarm) if cheap Nuclear fusion is available?

Forward onpremise logs to Kinesis Firehose

Code in Node.js AWS Lambda Package will not call putRecord() for Adding Data into AWS Kinesis Firehose StreamKinesis input stream into LogstashFirehose to S3: What happens to data after unsuccessful tries for 24 hoursIssues with streaming data to AWS Kinesis Firehose from PythonKinesis Analytics Destination Guidance: Lambda vs Kinesis Stream to LambdaKinesis Agent reports error from python stream dataSetting up write access of s3 owned by one aws account to firehose stream owned by another aws accountKinesis FireHose vs Kinesis Data Stream loosing dataResourceNotFoundException while adding data to Kinesis Firehose stream using Lambda

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;

0

For my application logging I am planning to forward the log from my on-premise servers to AWS Kinesis firehose. I am trying to use Logstash and the log-stash-output-plugin
https://github.com/samcday/logstash-output-kinesis

But I think this plugin requires to forward the log data to Kinesis Data Stream and then we can create a pipe to Kinesis Firehose. If we point to the Kinesis Firehose Delivery stream then we are getting the error -
ResourceNotFoundException Message: Stream test under account XXXX not found

But if we point the log-stash to Kinesis data stream and then pipe it to Kinesis Firehose then it is working fine.

Have anyone used this log-stashoutput plugin to forward the log data directly to Kinesis Firehose.

amazon-kinesis amazon-kinesis-firehose

share|improve this question

edited Mar 25 at 10:53

bugfreerammohan

1,3451215

asked Mar 25 at 8:43

DeveloperDeveloper

388

add a comment | 

0

For my application logging I am planning to forward the log from my on-premise servers to AWS Kinesis firehose. I am trying to use Logstash and the log-stash-output-plugin
https://github.com/samcday/logstash-output-kinesis

But I think this plugin requires to forward the log data to Kinesis Data Stream and then we can create a pipe to Kinesis Firehose. If we point to the Kinesis Firehose Delivery stream then we are getting the error -
ResourceNotFoundException Message: Stream test under account XXXX not found

But if we point the log-stash to Kinesis data stream and then pipe it to Kinesis Firehose then it is working fine.

Have anyone used this log-stashoutput plugin to forward the log data directly to Kinesis Firehose.

amazon-kinesis amazon-kinesis-firehose

share|improve this question

edited Mar 25 at 10:53

bugfreerammohan

1,3451215

asked Mar 25 at 8:43

DeveloperDeveloper

388

add a comment | 

For my application logging I am planning to forward the log from my on-premise servers to AWS Kinesis firehose. I am trying to use Logstash and the log-stash-output-plugin
https://github.com/samcday/logstash-output-kinesis

But I think this plugin requires to forward the log data to Kinesis Data Stream and then we can create a pipe to Kinesis Firehose. If we point to the Kinesis Firehose Delivery stream then we are getting the error -
ResourceNotFoundException Message: Stream test under account XXXX not found

But if we point the log-stash to Kinesis data stream and then pipe it to Kinesis Firehose then it is working fine.

Have anyone used this log-stashoutput plugin to forward the log data directly to Kinesis Firehose.

amazon-kinesis amazon-kinesis-firehose

share|improve this question

edited Mar 25 at 10:53

bugfreerammohan

1,3451215

asked Mar 25 at 8:43

DeveloperDeveloper

388

share|improve this question

edited Mar 25 at 10:53

bugfreerammohan

1,3451215

asked Mar 25 at 8:43

DeveloperDeveloper

388

share|improve this question

edited Mar 25 at 10:53

bugfreerammohan

1,3451215

asked Mar 25 at 8:43

DeveloperDeveloper

388

edited Mar 25 at 10:53

bugfreerammohan

1,3451215

edited Mar 25 at 10:53

bugfreerammohan

1,3451215

asked Mar 25 at 8:43

DeveloperDeveloper

388

asked Mar 25 at 8:43

DeveloperDeveloper

388

1 Answer
1

active

oldest

votes

0

logstash-output-kinesis uses Kinesis Producer library(KPL) to write data to Kinesis streams. As of now KPL does not support writing to Firehose directly.
- https://github.com/awslabs/amazon-kinesis-producer/issues/29

Also Firehose and Kinesis have different service endpoints and in logstash-output-kinesis project it seems there is only the option to set the kinesis endpoint.
So even if you specify a valid existing Firehose name in the config, since the endpoint set is that of AWS kinesis, you should expect to see "ResourceNotFoundException".
- https://github.com/samcday/logstash-output-kinesis/blob/acffae5beddcf00aa599c1fa75428f57bb7e7e30/lib/logstash/outputs/kinesis.rb#L55

share|improve this answer

answered Mar 27 at 23:55

HarshdevHarshdev

361

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55333985%2fforward-onpremise-logs-to-kinesis-firehose%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

logstash-output-kinesis uses Kinesis Producer library(KPL) to write data to Kinesis streams. As of now KPL does not support writing to Firehose directly.
- https://github.com/awslabs/amazon-kinesis-producer/issues/29

Also Firehose and Kinesis have different service endpoints and in logstash-output-kinesis project it seems there is only the option to set the kinesis endpoint.
So even if you specify a valid existing Firehose name in the config, since the endpoint set is that of AWS kinesis, you should expect to see "ResourceNotFoundException".
- https://github.com/samcday/logstash-output-kinesis/blob/acffae5beddcf00aa599c1fa75428f57bb7e7e30/lib/logstash/outputs/kinesis.rb#L55

share|improve this answer

answered Mar 27 at 23:55

HarshdevHarshdev

361

add a comment | 

0

logstash-output-kinesis uses Kinesis Producer library(KPL) to write data to Kinesis streams. As of now KPL does not support writing to Firehose directly.
- https://github.com/awslabs/amazon-kinesis-producer/issues/29

Also Firehose and Kinesis have different service endpoints and in logstash-output-kinesis project it seems there is only the option to set the kinesis endpoint.
So even if you specify a valid existing Firehose name in the config, since the endpoint set is that of AWS kinesis, you should expect to see "ResourceNotFoundException".
- https://github.com/samcday/logstash-output-kinesis/blob/acffae5beddcf00aa599c1fa75428f57bb7e7e30/lib/logstash/outputs/kinesis.rb#L55

share|improve this answer

answered Mar 27 at 23:55

HarshdevHarshdev

361

add a comment | 

logstash-output-kinesis uses Kinesis Producer library(KPL) to write data to Kinesis streams. As of now KPL does not support writing to Firehose directly.
- https://github.com/awslabs/amazon-kinesis-producer/issues/29

Also Firehose and Kinesis have different service endpoints and in logstash-output-kinesis project it seems there is only the option to set the kinesis endpoint.
So even if you specify a valid existing Firehose name in the config, since the endpoint set is that of AWS kinesis, you should expect to see "ResourceNotFoundException".
- https://github.com/samcday/logstash-output-kinesis/blob/acffae5beddcf00aa599c1fa75428f57bb7e7e30/lib/logstash/outputs/kinesis.rb#L55

share|improve this answer

answered Mar 27 at 23:55

HarshdevHarshdev

361

share|improve this answer

answered Mar 27 at 23:55

HarshdevHarshdev

361

answered Mar 27 at 23:55

HarshdevHarshdev

361

answered Mar 27 at 23:55

HarshdevHarshdev

361