Styjun

Shouldn't countries like Russia and Canada support global warming?

Examples of proofs by making reduction to a finite set

How to make a bold sparkline in Google Sheets?

Would it be unbalanced to increase a druid's number of uses of Wild Shape based on level?

Impossible Scrabble Words

How do certain apps show new notifications when internet access is restricted to them?

Why don't airports use arresting gears to recover energy from landing passenger planes?

shell script to check if input is a string/integer/float

How clean are pets?

Is the Dodge action perceptible to other characters?

How to control the output voltage of a solid state relay

Does a large scratch in an ND filter affect image quality?

Teleport everything in a large zone; or teleport all living things and make a lot of equipment disappear

Permutations in Disguise

Does my opponent need to prove his creature has morph?

Is it possible to format a USB from a live USB?

How do we know that black holes are spinning?

What was the motivation for the invention of electric pianos?

Can I travel to European countries with the Irish passport and without destination Visa?

How to publish superseding results without creating enemies

How to write characters doing illogical things in a believable way?

Seven Places at Once - Another Google Earth Challenge?

Python web-scraper to download table of transistor counts from Wikipedia

Why is this sentence grammatical?

How to identify the root cause of excess memory consumption in a Heroku Python deployment?

Heroku load average alarmingly highHeroku R14 Errors Immediately on RestartDeploy Django project on heroku (python 3.4.3)'ImportError: No module named ' , flask on herokuHeroku - No web process runningHeroku memory usage high in rails appModuleNotFoundError: No module named 'heroku'TypeError: __init__() got an unexpected keyword argument 'family_or_realsock'Flask-Socketio takes extremely long to connect on HerokuDjango App crashes when deployed to Heroku - Worker failed to boot

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

0

I have recently deployed a web app on Heroku (Free version). The deployment uses Python with libraries for plotly, dash, pandas and bootstrap. I would assume that this setup based on the size of all the files should only be in few 10s of MBs. However, it is consuming over 400 MB of RAM. How do I troubleshoot to identify the cause of this high memory (possibly a memory leak)?

I have tried using the heroku labs logs-runtime-metrics tool but the information is limited. I also tried to use the chrome developer tools but I was not able to reach anywhere.

heroku labs:enable log-runtime-metrics
heroku logs

The log looks something like this:

heroku[web.1]: source=web.1 dyno=heroku.XXXX sample#memory_total=458.20MB sample#memory_rss=431.30MB sample#memory_cache=26.89MB sample#memory_swap=0.00MB sample#memory_pgpgin=160223pages sample#memory_pgpgout=49568pages sample#memory_quota=512.00MB

As I mentioned above the memory_rss value which translated to the actual RAM consumption is too high for a python based function (close to the limit). Running the same process on my local machine peaks at around 180MB. Does the import of Python modules affect memory performance too much?

Looking for advice or help to point me in the right direction, please. Appreciate your help. Thanks.

---

EDIT 1: gunicorn --preload and improved codebase

I have tried to improve the memory optimization somewhat by using this hack where one can preload the application as explained here:
Gunicorn Preload

This is done by editing the Procfile to contain the following:

web: gunicorn hello:app --preload

I also did some edits in the way I was importing certain libraries and avoiding others whose purpose could be overridden. With all that my new logs show a decent improvement:

heroku[web.1]: source=web.1 dyno=heroku.XXXX sample#memory_total=272.63MB sample#memory_rss=248.45MB sample#memory_cache=24.18MB sample#memory_swap=0.00MB sample#memory_pgpgin=97938pages sample#memory_pgpgout=28145pages sample#memory_quota=512.00MB

More inputs are welcome!

python-3.x heroku plotly-dash

share|improve this question

edited Mar 28 at 15:15

SivamPillai

asked Mar 28 at 12:10

SivamPillaiSivamPillai

4644 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I don't have any tips or tools for you, but bear in mind that the memory consumption Heroku reports likely includes things like your web server (and maybe some system-level processes since everything is running in a container) that you probably aren't considering locally.
  
  – Chris
  Mar 28 at 12:49
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  That makes sense. I have done some minor optimizations (EDIT 1 in question) that have helped me. But this method also seems to slow the page loading I feel (have to measure it)
  
  – SivamPillai
  Mar 28 at 15:16
  

  
  
  
  

add a comment
 | 

0

I have recently deployed a web app on Heroku (Free version). The deployment uses Python with libraries for plotly, dash, pandas and bootstrap. I would assume that this setup based on the size of all the files should only be in few 10s of MBs. However, it is consuming over 400 MB of RAM. How do I troubleshoot to identify the cause of this high memory (possibly a memory leak)?

I have tried using the heroku labs logs-runtime-metrics tool but the information is limited. I also tried to use the chrome developer tools but I was not able to reach anywhere.

heroku labs:enable log-runtime-metrics
heroku logs

The log looks something like this:

heroku[web.1]: source=web.1 dyno=heroku.XXXX sample#memory_total=458.20MB sample#memory_rss=431.30MB sample#memory_cache=26.89MB sample#memory_swap=0.00MB sample#memory_pgpgin=160223pages sample#memory_pgpgout=49568pages sample#memory_quota=512.00MB

As I mentioned above the memory_rss value which translated to the actual RAM consumption is too high for a python based function (close to the limit). Running the same process on my local machine peaks at around 180MB. Does the import of Python modules affect memory performance too much?

Looking for advice or help to point me in the right direction, please. Appreciate your help. Thanks.

---

EDIT 1: gunicorn --preload and improved codebase

I have tried to improve the memory optimization somewhat by using this hack where one can preload the application as explained here:
Gunicorn Preload

This is done by editing the Procfile to contain the following:

web: gunicorn hello:app --preload

I also did some edits in the way I was importing certain libraries and avoiding others whose purpose could be overridden. With all that my new logs show a decent improvement:

heroku[web.1]: source=web.1 dyno=heroku.XXXX sample#memory_total=272.63MB sample#memory_rss=248.45MB sample#memory_cache=24.18MB sample#memory_swap=0.00MB sample#memory_pgpgin=97938pages sample#memory_pgpgout=28145pages sample#memory_quota=512.00MB

More inputs are welcome!

python-3.x heroku plotly-dash

share|improve this question

edited Mar 28 at 15:15

SivamPillai

asked Mar 28 at 12:10

SivamPillaiSivamPillai

4644 bronze badges

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I don't have any tips or tools for you, but bear in mind that the memory consumption Heroku reports likely includes things like your web server (and maybe some system-level processes since everything is running in a container) that you probably aren't considering locally.
  
  – Chris
  Mar 28 at 12:49
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  That makes sense. I have done some minor optimizations (EDIT 1 in question) that have helped me. But this method also seems to slow the page loading I feel (have to measure it)
  
  – SivamPillai
  Mar 28 at 15:16
  

  
  
  
  

add a comment
 | 

I have recently deployed a web app on Heroku (Free version). The deployment uses Python with libraries for plotly, dash, pandas and bootstrap. I would assume that this setup based on the size of all the files should only be in few 10s of MBs. However, it is consuming over 400 MB of RAM. How do I troubleshoot to identify the cause of this high memory (possibly a memory leak)?

I have tried using the heroku labs logs-runtime-metrics tool but the information is limited. I also tried to use the chrome developer tools but I was not able to reach anywhere.

heroku labs:enable log-runtime-metrics
heroku logs

The log looks something like this:

heroku[web.1]: source=web.1 dyno=heroku.XXXX sample#memory_total=458.20MB sample#memory_rss=431.30MB sample#memory_cache=26.89MB sample#memory_swap=0.00MB sample#memory_pgpgin=160223pages sample#memory_pgpgout=49568pages sample#memory_quota=512.00MB

As I mentioned above the memory_rss value which translated to the actual RAM consumption is too high for a python based function (close to the limit). Running the same process on my local machine peaks at around 180MB. Does the import of Python modules affect memory performance too much?

Looking for advice or help to point me in the right direction, please. Appreciate your help. Thanks.

---

EDIT 1: gunicorn --preload and improved codebase

I have tried to improve the memory optimization somewhat by using this hack where one can preload the application as explained here:
Gunicorn Preload

This is done by editing the Procfile to contain the following:

web: gunicorn hello:app --preload

I also did some edits in the way I was importing certain libraries and avoiding others whose purpose could be overridden. With all that my new logs show a decent improvement:

heroku[web.1]: source=web.1 dyno=heroku.XXXX sample#memory_total=272.63MB sample#memory_rss=248.45MB sample#memory_cache=24.18MB sample#memory_swap=0.00MB sample#memory_pgpgin=97938pages sample#memory_pgpgout=28145pages sample#memory_quota=512.00MB

More inputs are welcome!

python-3.x heroku plotly-dash

share|improve this question

edited Mar 28 at 15:15

SivamPillai

asked Mar 28 at 12:10

SivamPillaiSivamPillai

4644 bronze badges

heroku labs:enable log-runtime-metrics
heroku logs

heroku[web.1]: source=web.1 dyno=heroku.XXXX sample#memory_total=458.20MB sample#memory_rss=431.30MB sample#memory_cache=26.89MB sample#memory_swap=0.00MB sample#memory_pgpgin=160223pages sample#memory_pgpgout=49568pages sample#memory_quota=512.00MB

web: gunicorn hello:app --preload

heroku[web.1]: source=web.1 dyno=heroku.XXXX sample#memory_total=272.63MB sample#memory_rss=248.45MB sample#memory_cache=24.18MB sample#memory_swap=0.00MB sample#memory_pgpgin=97938pages sample#memory_pgpgout=28145pages sample#memory_quota=512.00MB

share|improve this question

edited Mar 28 at 15:15

SivamPillai

asked Mar 28 at 12:10

SivamPillaiSivamPillai

4644 bronze badges

share|improve this question

edited Mar 28 at 15:15

SivamPillai

asked Mar 28 at 12:10

SivamPillaiSivamPillai

4644 bronze badges

asked Mar 28 at 12:10

SivamPillaiSivamPillai

4644 bronze badges

asked Mar 28 at 12:10

SivamPillaiSivamPillai

4644 bronze badges