How can I save stream result into remote database via REST or anything easilySend byte array to storm kafka boltKafka Stream: output to a topic first or persist directly?Kafka streams.allMetadata() method returns empty listHow can I emit the result of storm botl to kafka brokers using kafkabolt?Streaming messages from one Kafka Cluster to anotherBest practices for Kafka streamsSpark Kafka Streaming - Send original timestamp rather than current timestampKafka Streams Stream Table Join - What if Key Doesn't Exist in Table?Connector Transforms Incorrectly Maps Partition KeyFlink outputs to Kafka, proper way to use KeyedSerializationSchema
Why did Gandalf use a sword against the Balrog?
Does Molecular Weight of a Gas affect its lifting properties at the same velocity over the same wing?
Not going forward with internship interview process
How can Radagast come across Gandalf and Thorin's company?
Safest way to store environment variable value in a file
Plotting octahedron inside the sphere and sphere inside the cube
Why is the result of ('b'+'a'+ + 'a' + 'a').toLowerCase() 'banana'?
visible indication that a cell is not evaluatable
Why are Tucker and Malcolm not dead?
If "more guns less crime", how do gun advocates explain that the EU has less crime than the US?
Is this curved text blend possible in Illustrator?
How to take the beginning and end parts of a list with simpler syntax?
Can sampling rate be a floating point number?
Understanding the point of a kölsche Witz
If clocks themselves are based on light signals, wouldn't we expect the measured speed of light to always be the same constant?
What ability do tools use?
How far did Gandalf and the Balrog drop from the bridge in Moria?
Is it okay for a ticket seller in the USA to refuse to give you your change, keep it for themselves and claim it's a tip?
Why aren't rainbows blurred-out into nothing after they are produced?
Heating Margarine in Pan = loss of calories?
TEMPO: play a sound in animated GIF/PDF/SVG
Why are Gatwick's runways too close together?
How to reduce Sinas Chinam
Plotting of residuals by fitted values vs. sorted by fitted values
How can I save stream result into remote database via REST or anything easily
Send byte array to storm kafka boltKafka Stream: output to a topic first or persist directly?Kafka streams.allMetadata() method returns empty listHow can I emit the result of storm botl to kafka brokers using kafkabolt?Streaming messages from one Kafka Cluster to anotherBest practices for Kafka streamsSpark Kafka Streaming - Send original timestamp rather than current timestampKafka Streams Stream Table Join - What if Key Doesn't Exist in Table?Connector Transforms Incorrectly Maps Partition KeyFlink outputs to Kafka, proper way to use KeyedSerializationSchema
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have examined confluent kafka stream wordcount and anomaly detection example. In these example result is written to a topic. Instead of this How can I save the result iinto remote database via REST or anything easily and fastly. Are there any structure in confluent platform
Code example:
// instead of code, send remote database
wordCounts.toStream().to("streams-wordcount-output", Produced.with(stringSerde, longSerde));
apache-kafka apache-kafka-connect
edited Mar 27 at 10:58
Robin Moffatt
10.4k17 silver badges31 bronze badges
asked Mar 27 at 7:35
validatorvalidator
83 bronze badges
add a comment |
I have examined confluent kafka stream wordcount and anomaly detection example. In these example result is written to a topic. Instead of this How can I save the result iinto remote database via REST or anything easily and fastly. Are there any structure in confluent platform
Code example:
// instead of code, send remote database
wordCounts.toStream().to("streams-wordcount-output", Produced.with(stringSerde, longSerde));
apache-kafka apache-kafka-connect
edited Mar 27 at 10:58
Robin Moffatt
10.4k17 silver badges31 bronze badges
asked Mar 27 at 7:35
validatorvalidator
83 bronze badges
add a comment |
I have examined confluent kafka stream wordcount and anomaly detection example. In these example result is written to a topic. Instead of this How can I save the result iinto remote database via REST or anything easily and fastly. Are there any structure in confluent platform
Code example:
// instead of code, send remote database
wordCounts.toStream().to("streams-wordcount-output", Produced.with(stringSerde, longSerde));
apache-kafka apache-kafka-connect
edited Mar 27 at 10:58
Robin Moffatt
10.4k17 silver badges31 bronze badges
asked Mar 27 at 7:35
validatorvalidator
83 bronze badges
I have examined confluent kafka stream wordcount and anomaly detection example. In these example result is written to a topic. Instead of this How can I save the result iinto remote database via REST or anything easily and fastly. Are there any structure in confluent platform
Code example:
// instead of code, send remote database
wordCounts.toStream().to("streams-wordcount-output", Produced.with(stringSerde, longSerde));
apache-kafka apache-kafka-connect
apache-kafka apache-kafka-connect
edited Mar 27 at 10:58
Robin Moffatt
10.4k17 silver badges31 bronze badges
asked Mar 27 at 7:35
validatorvalidator
83 bronze badges
edited Mar 27 at 10:58
Robin Moffatt
10.4k17 silver badges31 bronze badges
asked Mar 27 at 7:35
validatorvalidator
83 bronze badges
edited Mar 27 at 10:58
Robin Moffatt
10.4k17 silver badges31 bronze badges
edited Mar 27 at 10:58
Robin Moffatt
10.4k17 silver badges31 bronze badges
edited Mar 27 at 10:58
Robin Moffatt
10.4k17 silver badges31 bronze badges
10.4k17 silver badges31 bronze badges
asked Mar 27 at 7:35
validatorvalidator
83 bronze badges
asked Mar 27 at 7:35
validatorvalidator
83 bronze badges
asked Mar 27 at 7:35
validatorvalidator
83 bronze badges
83 bronze badges
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
The usual pattern here is to write the results of your stream processing to a Kafka topic, and then use Kafka Connect to stream that topic to anywhere you want to persist the data to. Kafka Connect is part of Apache Kafka, and there are numerous connectors, including kafka-connect-jdbc for writing data to (and from) databases.
If you write directly from your streams application to the database you're unnecessarily tying together your processing and your storage. If the database is offline or unreachable your stream processing has to handle that. Instead decouple the two, and Kafka Connect will handle unreachable database etc.
answered Mar 27 at 10:58
Robin MoffattRobin Moffatt
10.4k17 silver badges31 bronze badges
Thank you so much Robin. I understand you said. I start to search kafka-connect-jdbc.
– validator
Mar 27 at 11:25
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55371961%2fhow-can-i-save-stream-result-into-remote-database-via-rest-or-anything-easily%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The usual pattern here is to write the results of your stream processing to a Kafka topic, and then use Kafka Connect to stream that topic to anywhere you want to persist the data to. Kafka Connect is part of Apache Kafka, and there are numerous connectors, including kafka-connect-jdbc for writing data to (and from) databases.
If you write directly from your streams application to the database you're unnecessarily tying together your processing and your storage. If the database is offline or unreachable your stream processing has to handle that. Instead decouple the two, and Kafka Connect will handle unreachable database etc.
answered Mar 27 at 10:58
Robin MoffattRobin Moffatt
10.4k17 silver badges31 bronze badges
Thank you so much Robin. I understand you said. I start to search kafka-connect-jdbc.
– validator
Mar 27 at 11:25
add a comment |
The usual pattern here is to write the results of your stream processing to a Kafka topic, and then use Kafka Connect to stream that topic to anywhere you want to persist the data to. Kafka Connect is part of Apache Kafka, and there are numerous connectors, including kafka-connect-jdbc for writing data to (and from) databases.
If you write directly from your streams application to the database you're unnecessarily tying together your processing and your storage. If the database is offline or unreachable your stream processing has to handle that. Instead decouple the two, and Kafka Connect will handle unreachable database etc.
answered Mar 27 at 10:58
Robin MoffattRobin Moffatt
10.4k17 silver badges31 bronze badges
Thank you so much Robin. I understand you said. I start to search kafka-connect-jdbc.
– validator
Mar 27 at 11:25
add a comment |
The usual pattern here is to write the results of your stream processing to a Kafka topic, and then use Kafka Connect to stream that topic to anywhere you want to persist the data to. Kafka Connect is part of Apache Kafka, and there are numerous connectors, including kafka-connect-jdbc for writing data to (and from) databases.
If you write directly from your streams application to the database you're unnecessarily tying together your processing and your storage. If the database is offline or unreachable your stream processing has to handle that. Instead decouple the two, and Kafka Connect will handle unreachable database etc.
answered Mar 27 at 10:58
Robin MoffattRobin Moffatt
10.4k17 silver badges31 bronze badges
The usual pattern here is to write the results of your stream processing to a Kafka topic, and then use Kafka Connect to stream that topic to anywhere you want to persist the data to. Kafka Connect is part of Apache Kafka, and there are numerous connectors, including kafka-connect-jdbc for writing data to (and from) databases.
If you write directly from your streams application to the database you're unnecessarily tying together your processing and your storage. If the database is offline or unreachable your stream processing has to handle that. Instead decouple the two, and Kafka Connect will handle unreachable database etc.
answered Mar 27 at 10:58
Robin MoffattRobin Moffatt
10.4k17 silver badges31 bronze badges
answered Mar 27 at 10:58
Robin MoffattRobin Moffatt
10.4k17 silver badges31 bronze badges
answered Mar 27 at 10:58
Robin MoffattRobin Moffatt
10.4k17 silver badges31 bronze badges
answered Mar 27 at 10:58
Robin MoffattRobin Moffatt
10.4k17 silver badges31 bronze badges
10.4k17 silver badges31 bronze badges
Thank you so much Robin. I understand you said. I start to search kafka-connect-jdbc.
– validator
Mar 27 at 11:25
add a comment |
Thank you so much Robin. I understand you said. I start to search kafka-connect-jdbc.
– validator
Mar 27 at 11:25
Thank you so much Robin. I understand you said. I start to search kafka-connect-jdbc.
– validator
Mar 27 at 11:25
Thank you so much Robin. I understand you said. I start to search kafka-connect-jdbc.
– validator
Mar 27 at 11:25
add a comment |
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55371961%2fhow-can-i-save-stream-result-into-remote-database-via-rest-or-anything-easily%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
