Struggling with a Prometheus percentile query on jvm statsPrometheus - Docker/JVM MonitoringPrometheus Startup queryPrometheus nested alert query in prometheusQuery span duration in PrometheusPrometheus query returns invalid valueHow to measure HTTP Latency with Percentile in PrometheusAverage Memory Usage Query - PrometheusPrometheus - debugging slow queryHow to monitor pool stats in Atomikos with prometheusPrometheus/Grafana plot wait time percentiles for jobs

Are randomly-generated passwords starting with "a" less secure?

Is there a way to perform "if" check only once instead of every iteration in a loop?

Where or how can I find what interfaces an out of the box Apex class implements?

Was the Ford Model T black because of the speed black paint dries?

Credit union holding car note, refuses to provide details of how payments have been applied

Steampunk helicopter

Is purchasing foreign currency before going abroad a losing proposition?

QGIS Welcome page: What is 'pin to list' for?

Do native speakers use ZVE or CPU?

Crowbar circuit causes unexpected behavior for op amp circuit

affect vs effect

Was adding milk to tea started to reduce employee tea break time?

(async () => )(); what is this?

What would the EU do if an EU member declared war on another EU member?

Why are Hobbits so fond of mushrooms?

What would be the ideal melee weapon made of "Phase Metal"?

Keep milk (or milk alternative) for a day without a fridge

Dropping outliers based on "2.5 times the RMSE"

Can I call 112 to check a police officer's identity in the Czech Republic?

Referring to different instances of the same character in time travel

What is temperature on a quantum level

What is the difference between logical consistency and logical entailment in deductive logic?

Is an acid a salt or not?

Why did my rum cake turn black?



Struggling with a Prometheus percentile query on jvm stats


Prometheus - Docker/JVM MonitoringPrometheus Startup queryPrometheus nested alert query in prometheusQuery span duration in PrometheusPrometheus query returns invalid valueHow to measure HTTP Latency with Percentile in PrometheusAverage Memory Usage Query - PrometheusPrometheus - debugging slow queryHow to monitor pool stats in Atomikos with prometheusPrometheus/Grafana plot wait time percentiles for jobs






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








0















I'm writing a query to get the 95th percentile on the rate of change in the jvm_gc_collection_seconds_count. I want to trigger an alert when there is an unexpected spike. The purpose is to send me an alert when the JVM is working harder than usual.



I am using jvm_memory_bytes_used for testing as the rate of the GC does not show any anomalies right now. Below is an example of the data I get back from the jvm_memory_bytes_used. Should the first top two blue spikes not be returned as data points? The bottom red line should surely be picked up as an anomaly.



sum of the rate of jvm_memory_bytes_used over 30 minutes



I have calculated the sum of the rate over 30 minutes grouped by instance. With that I try to get the 95th percentile.



histogram_quantile(0.95, sum(rate(jvm_memory_bytes_used[30m])) by (instance, le))


I've tried lowering the percentile which has no effect.



  1. Am I using the correct variable to calculate the GC stats?

  2. How can I get the histogram_quantile to produce results?

  3. Am I taking the right approach using a percentile to achieve my goal?









share|improve this question




























    0















    I'm writing a query to get the 95th percentile on the rate of change in the jvm_gc_collection_seconds_count. I want to trigger an alert when there is an unexpected spike. The purpose is to send me an alert when the JVM is working harder than usual.



    I am using jvm_memory_bytes_used for testing as the rate of the GC does not show any anomalies right now. Below is an example of the data I get back from the jvm_memory_bytes_used. Should the first top two blue spikes not be returned as data points? The bottom red line should surely be picked up as an anomaly.



    sum of the rate of jvm_memory_bytes_used over 30 minutes



    I have calculated the sum of the rate over 30 minutes grouped by instance. With that I try to get the 95th percentile.



    histogram_quantile(0.95, sum(rate(jvm_memory_bytes_used[30m])) by (instance, le))


    I've tried lowering the percentile which has no effect.



    1. Am I using the correct variable to calculate the GC stats?

    2. How can I get the histogram_quantile to produce results?

    3. Am I taking the right approach using a percentile to achieve my goal?









    share|improve this question
























      0












      0








      0








      I'm writing a query to get the 95th percentile on the rate of change in the jvm_gc_collection_seconds_count. I want to trigger an alert when there is an unexpected spike. The purpose is to send me an alert when the JVM is working harder than usual.



      I am using jvm_memory_bytes_used for testing as the rate of the GC does not show any anomalies right now. Below is an example of the data I get back from the jvm_memory_bytes_used. Should the first top two blue spikes not be returned as data points? The bottom red line should surely be picked up as an anomaly.



      sum of the rate of jvm_memory_bytes_used over 30 minutes



      I have calculated the sum of the rate over 30 minutes grouped by instance. With that I try to get the 95th percentile.



      histogram_quantile(0.95, sum(rate(jvm_memory_bytes_used[30m])) by (instance, le))


      I've tried lowering the percentile which has no effect.



      1. Am I using the correct variable to calculate the GC stats?

      2. How can I get the histogram_quantile to produce results?

      3. Am I taking the right approach using a percentile to achieve my goal?









      share|improve this question














      I'm writing a query to get the 95th percentile on the rate of change in the jvm_gc_collection_seconds_count. I want to trigger an alert when there is an unexpected spike. The purpose is to send me an alert when the JVM is working harder than usual.



      I am using jvm_memory_bytes_used for testing as the rate of the GC does not show any anomalies right now. Below is an example of the data I get back from the jvm_memory_bytes_used. Should the first top two blue spikes not be returned as data points? The bottom red line should surely be picked up as an anomaly.



      sum of the rate of jvm_memory_bytes_used over 30 minutes



      I have calculated the sum of the rate over 30 minutes grouped by instance. With that I try to get the 95th percentile.



      histogram_quantile(0.95, sum(rate(jvm_memory_bytes_used[30m])) by (instance, le))


      I've tried lowering the percentile which has no effect.



      1. Am I using the correct variable to calculate the GC stats?

      2. How can I get the histogram_quantile to produce results?

      3. Am I taking the right approach using a percentile to achieve my goal?






      prometheus






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Mar 26 at 4:26









      cbillowescbillowes

      2,0182 gold badges21 silver badges39 bronze badges




      2,0182 gold badges21 silver badges39 bronze badges






















          0






          active

          oldest

          votes










          Your Answer






          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "1"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55349850%2fstruggling-with-a-prometheus-percentile-query-on-jvm-stats%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes




          Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.







          Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.



















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55349850%2fstruggling-with-a-prometheus-percentile-query-on-jvm-stats%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          How to preserve electronics (computers, tablets and phones) for hundreds of years Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern) The network's official Twitter account is up and running again. What content…How might modern humans leave a message for 50,000 years?How would an aquatic race develop computers?How do we make computers care for human life?Keeping supplies for 1000 yearsWhat will cell phones become in 100 years?What body armor protects against a laser?Could a society attain 1800s technology if limited to using copper and bronze?Electricity power (magic) and electronicsScavenging metal resources in post-apocalypse after 100 yearsWhy would precursors create devices that can survive and still work after hundreds of years?

          155 수학 과학 기타 둘러보기 메뉴eh추가해eh문서를 완성해

          Convert JSON to MapSimplest way to convert a string representing JSON array into Java array List of key value pairsHow to take String JSON values into HashMapConvert a JSON String to a HashMapCreating BSON object from JSON stringJSONObject - How to get a value ?Better Map ConstructorReading JSON from Url javaUpdate columns if input values are not null otherwise ignore and keep the existing values of column in databaseHow to store object represented as Json string into Couchbase Lite?Go JSON into MapHow do I efficiently iterate over each entry in a Java Map?Sort a Map<Key, Value> by valuesHow to round a number to n decimal places in JavaCan comments be used in JSON?How do I read / convert an InputStream into a String in Java?How can I pretty-print JSON in a shell script?What is the correct JSON content type?How can I deserialize JSON to a simple Dictionary<string,string> in ASP.NET?Why does Google prepend while(1); to their JSON responses?Parse JSON in JavaScript?