Multi tool use

Clash Royale CLAN TAG

saving cProfile results to readable external file

I am using cProfile try to profile my codes:

cProfile

pr = cProfile.Profile()
pr.enable()
my_func() # the code I want to profile
pr.disable()
pr.print_stats()


However, the results are too long and cannot be fully displayed in the Spyder terminal (the function calls which take the longest time to run cannot be seen...). I also tried saving the results using cProfile.run('my_func()','profile_results'), but the output file is not in human readable format (tried with and without .txt suffix).

cProfile.run('my_func()','profile_results')

.txt

So my question is how can I save the profiling results to an external file which is human readable (like in a .txt format with all the words properly displayed)?

dump_stats

pstats.Stats

1 Answer
1

I'm not sure if that's what you need. I can recommend to use dump_stats + pstats.Stats. Here is an example how to use it. Structure of files:

# test_ex.py - just a small web app
import cProfile
import json
from functools import wraps
from flask import Flask

from example.mod1 import func1
from example.mod2 import func2

app = Flask(__name__)

# profiling decorator
def profiling():

def _profiling(f):
@wraps(f)
def __profiling(*rgs, **kwargs):
pr = cProfile.Profile()
pr.enable()

result = f(*rgs, **kwargs)

pr.disable()
# save stats into file
pr.dump_stats('profile_dump')

return result
return __profiling
return _profiling

# demonstration route with profiler
@app.route('/test')
@profiling()
def test():
counter = func1()
dict_data = func2()
result = dict()

for key, val in dict_data.items():
result[key] = val + counter

return json.dumps(result)


if __name__ == '__main__':
app.run(debug=True, port=8083)


example package - let's imagine that this is some kind of application logic.

# example.mod1
def func1():
counter = 0

for i in range(100000):
counter += i

return counter

# example.mod2
def func2():
res = dict()

for i in range(300000):
res['key_' + str(i)] = i

return res


Now let's run server(python3 test_ex.py) and open http://localhost:8083/test. After a few seconds you will see long json. After that you will see profile_dump file in project folder. Now run python live interpreter in project folder and print our dump using pstats:

python3 test_ex.py

http://localhost:8083/test

import pstats
p = pstats.Stats('profile_dump')
# skip strip_dirs() if you want to see full path's
p.strip_dirs().print_stats()


Also you can easy sorting results:

p.strip_dirs().sort_stats('tottime').print_stats()
p.strip_dirs().sort_stats('cumulative').print_stats()
p.strip_dirs().sort_stats().print_stats('mod1')


Hope this helps.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.