Is it possible to calculate the disk space consumed by a project?
Thanks!
Hi Ed
You can get a rough estimate using the calculate_physical_size command:
python manage.py calculate_physical_size [domain]
This will produce a rough estimate and only includes forms and cases (not applications, users etc) and also does not take into consideration form attachment.
1 Like
Actually that's pretty useful, thanks Simon. The app itself and users take up negligible space from my experience. I don't think we're doing too many attachments if any, so all good there. Ta!
I may be doing something wrong but I'm getting an error running manage.py
(cchq) xxxxx@monolith:/home/cchq/www/monolith/releases/2022-03-17_04.54$ python manage.py calculate_physical_size [domain-name]
Traceback (most recent call last):
File "manage.py", line 7, in
from psycogreen.gevent import patch_psycopg
ModuleNotFoundError: No module named 'psycogreen'
EDIT silly mistake - I hadn't activated the virtual environment. After activating with
cd /home/cchq/www/monolith/releases/2022-03-08_18.52
source python_env-3.9/bin/activate
I ran the command but got this:
Traceback (most recent call last):
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/manage.py", line 208, in <module>
main()
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/manage.py", line 43, in main
execute_from_command_line(sys.argv)
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/core/management/__init__.py", line 325, in execute
settings.INSTALLED_APPS
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/conf/__init__.py", line 79, in __getattr__
self._setup(name)
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/conf/__init__.py", line 66, in _setup
self._wrapped = Settings(settings_module)
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/conf/__init__.py", line 157, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/settings.py", line 1126, in <module>
from localsettings import *
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 846, in exec_module
File "<frozen importlib._bootstrap_external>", line 982, in get_code
File "<frozen importlib._bootstrap_external>", line 1039, in get_data
PermissionError: [Errno 13] Permission denied: '/home/cchq/www/monolith/releases/2022-03-08_18.52/localsettings.py'
That's unusual, better check what the permissions of that file are.
/home/cchq/www/monolith/releases/2022-03-08_18.52/localsettings.py:
-rw-r--r-- 1 cchq cchq 4488 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/dev_settings.py
-rw-r--r-- 1 cchq cchq 2336 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/fabfile.py
-rw-r--r-- 1 cchq cchq 517 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/get_resource_versions.py
-rw-r--r-- 1 cchq cchq 2491 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/localsettings.example.py
-rw------- 1 cchq cchq 17188 Mar 8 18:53 /home/cchq/www/monolith/releases/2022-03-08_18.52/localsettings.py
-rwxr-xr-x 1 cchq cchq 6466 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/manage.py
-rw-r--r-- 1 cchq cchq 0 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/repeater_fire.py
-rw-r--r-- 1 cchq cchq 1713 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/scent.py
-rwxr-xr-x 1 cchq cchq 73543 Mar 8 19:13 /home/cchq/www/monolith/releases/2022-03-08_18.52/settings.py
-rw-r--r-- 1 cchq cchq 9723 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/settingshelper.py
-rw-r--r-- 1 cchq cchq 3988 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/testsettings.py
-rw-r--r-- 1 cchq cchq 9164 Mar 8 18:52 /home/cchq/www/monolith/releases/2022-03-08_18.52/urls.py
Edit after a chmod 644 on localsettings.py I ran it again and get this:
(python_env-3.9) ccc@monolith:/home/cchq/www/monolith/releases/2022-03-08_18.52$ python manage.py calculate_physical_size mydomain
Traceback (most recent call last):
File "/usr/lib/python3.9/logging/config.py", line 564, in configure
handler = self.configure_handler(handlers[name])
File "/usr/lib/python3.9/logging/config.py", line 745, in configure_handler
result = factory(**kwargs)
File "/usr/lib/python3.9/logging/handlers.py", line 153, in __init__
BaseRotatingHandler.__init__(self, filename, mode, encoding=encoding,
File "/usr/lib/python3.9/logging/handlers.py", line 58, in __init__
logging.FileHandler.__init__(self, filename, mode=mode,
File "/usr/lib/python3.9/logging/__init__.py", line 1146, in __init__
StreamHandler.__init__(self, self._open())
File "/usr/lib/python3.9/logging/__init__.py", line 1175, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding,
PermissionError: [Errno 13] Permission denied: '/home/cchq/www/monolith/log/monolith-commcarehq.accounting.log'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/manage.py", line 208, in <module>
main()
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/manage.py", line 43, in main
execute_from_command_line(sys.argv)
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/core/management/__init__.py", line 357, in execute
django.setup()
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/manage.py", line 161, in _setup_once
_setup_once.setup(*args, **kw)
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/__init__.py", line 19, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "/home/cchq/www/monolith/releases/2022-03-08_18.52/python_env-3.9/lib/python3.9/site-packages/django/utils/log.py", line 76, in configure_logging
logging_config_func(logging_settings)
File "/usr/lib/python3.9/logging/config.py", line 809, in dictConfig
dictConfigClass(config).configure()
File "/usr/lib/python3.9/logging/config.py", line 571, in configure
raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'accountinglog'
Checking permissions on /home/cchq/www/monolith/log/monolith-commcarehq.accounting.log...
-rw-r--r-- 1 cchq cchq 524717 May 24 05:00 /home/cchq/www/monolith/log/monolith-commcarehq.accounting.log
Are you executing these commands as the cchq user?
You can 'login' as that use as follows:
sudo -iu cchq
1 Like
Haha... indeed! I'm seeing my user (called ccc) and of course it's cchq who has rights. Ugh... long day! Will report back shortly, thanks Simon
EDIT spot on Simon, thanks a ton - problem solved!
mydomain has 985 forms, taking up approximately 10307519 bytes (9.83 MB) of physical space.
mydomain has 419 cases, taking up approximately 3046509 bytes (2.91 MB) of physical space.
These domains have a total of 985 forms, taking up approximately 10307519 bytes (9.83 MB) of physical space.
These domains have a total of 419 cases, taking up approximately 3046509 bytes (2.91 MB) of physical space.
Great! I'm glad that's resolved.
1 Like