In rapidsms.ini, media root is defined like so:
media_root=file:///tmp
Is there any Commcare specific code that relies on this? A grep didn't
come up with anything obvious. Django seems to be confused by this,
appending it to the project root and then throwing errors like:
IOError: [Errno 2] No such file or directory: u'/Users/niryariv/
Projects/commcare-hq/file:/tmp/data/photos/hoover.jpg'
The same goes for:
cache_backend=file:///tmp/django_cache
hey nir,
so...the cache_backend is for file:///tmp/ is fairly standard django
practice. that ought to just autogenerate the right files...if you need to
really makei t work, just make sure those paths exist and your user
executing them has the correct RW permissions.
regarding the MEDIA_ROOT and MEDIA_URL, it's a rather crazy bit. but i
think we have a workaround for that.
Long explanation
So, quick answer is that thereis no commcare specific rule because the way
the rapidsms bootstrapper work, it hasn't really come up because the
rapidsms webapp/urls.py walks through all the apps/ directories and looks
for a static/ directory to map using the built in django debug server.
now, in a regular app, you probably can get away with this, but with
ImageKit or any other django app that needs to deal with files this becomes
problematic. Here's why.
when rapidsms maps a static directory (to server, say, css and javascript
and static images)
it'll map it like this:
/apps/myapp/static/css/somestyle.css
/apps/myapp/static/img/image.jpg
built into the MEDIA_ROOT of django is the implicit /
so, when you see the image or stylesheet, it won't use the full hostname,
it'll just do relative pathing
so, <img src="/static/myapp/img/image.jpg>, say, instead of
Anyway, the issue here is notice how the paths flip from myapp/static/ to
static/myapp?
I'm guessing you're getting a bunch of errors with regard to Suspicious
Operation deep in the django file management modules. This is because the
django file manager checks the MEDIA_ROOT with the end paths of the media
URL and the actual file url.
1: when we serve our stuff up directly in apache, we symlink all the
static/ directories to a single staticmedia/ folder in a separate location.
we set the root of that folder to MEDIA_ROOT and the file suspicious
operations go away, as the path flipping no longer happens.
To see this in action look at setapache_static.py and also look for the
string USE_DJANGO_STATIC_SERVER to see how this works.
2: For ImageKit in particular, you can also manually add its own path to
the urls to serve it up in a directory structure that the django system will
like.
settings.py:
AUX_MEDIA_DIRS = {
'imagekit':
os.path.join(filepath,'somedir','imagekitmedia','media','imagekit'),
}
urls.py modifications
if hasattr(settings,'AUX_MEDIA_DIRS'):
if hasattr(settings,'USE_DJANGO_STATIC_SERVER') and
settings.USE_DJANGO_STATIC_SERVER:
for dirname, path in settings.AUX_MEDIA_DIRS.items():
if os.path.exists(path):
urlpatterns += patterns("", url("^media/%s/(?P.*)$" %
dirname,
"django.views.static.serve", {"document_root":
path }
))
Note, that the filesystem path is now:
somedir/imagekitmedia/imagekit/media
and the actaual URL pattern is now
/media/imagekit/
so if the img src now says 
django will not throw errors because that file will directly exist in:
/somedir/imagekitmedia/media/imagekit/img/image.jpg
I think option 2 is probably the simplest way to move forward..but it might
require you to move some files around. The long term solution is that all
the apps static directories should mirror how the templates/ directories
function.
Dan
On Tue, Feb 2, 2010 at 4:33 PM, Nir Yariv niryariv@gmail.com wrote:
In rapidsms.ini, media root is defined like so:
media_root=file:///tmp
Is there any Commcare specific code that relies on this? A grep didn't
come up with anything obvious. Django seems to be confused by this,
appending it to the project root and then throwing errors like:IOError: [Errno 2] No such file or directory: u'/Users/niryariv/
Projects/commcare-hq/file:/tmp/data/photos/hoover.jpg'The same goes for:
cache_backend=file:///tmp/django_cache
--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
Hi Daniel,
After posting the message to the list yesterday I kept working on
this, and came up with the following solution:
media_root is commented out in local.ini (or it could be left in
with an empty value)
In apps/photos/urls.py I added the following mapping:
(r'^data/photos/(?P.*)$', 'django.views.static.serve',
{"document_root": 'data/photos/'}),
(Photos are stored under data/photos, next to the data/attachments etc
directories)
What do you think? It seems to work ok, and doesn't break anything
(though I might well be missing something..). Storing under data seems
to make sense, since that's where data from the xforms gets stored. If
you think we can go ahead with this I'll move 'data/photos' to an
export_path setting in local.ini, as in receiver/xformmanager.
Thanks,
Nir
> Anyway, the issue here is notice how the paths flip from myapp/static/ to
> static/myapp?
> I'm guessing you're getting a bunch of errors with regard to Suspicious
> Operation deep in the django file management modules. This is because the
> django file manager checks the MEDIA_ROOT with the end paths of the media
> URL and the actual file url.
> We have 2 suggested workarounds:
> 1: when we serve our stuff up directly in apache, we symlink all the
> static/ directories to a single staticmedia/ folder in a separate location.
> we set the root of that folder to MEDIA_ROOT and the file suspicious
> operations go away, as the path flipping no longer happens.
> To see this in action look at setapache_static.py and also look for the
> string USE_DJANGO_STATIC_SERVER to see how this works.
>
> 2: For ImageKit in particular, you can also manually add its own path to
> the urls to serve it up in a directory structure that the django system will
> like.
> settings.py:
> AUX_MEDIA_DIRS = {
> 'imagekit':
> os.path.join(filepath,'somedir','imagekitmedia','media','imagekit'),
>
> }
> urls.py modifications
> if hasattr(settings,'AUX_MEDIA_DIRS'):
> if hasattr(settings,'USE_DJANGO_STATIC_SERVER') and
> settings.USE_DJANGO_STATIC_SERVER:
> for dirname, path in settings.AUX_MEDIA_DIRS.items():
> if os.path.exists(path):
> urlpatterns += patterns("", url("^media/%s/(?P.*)$" %
> dirname,
> "django.views.static.serve", {"document_root":
> path }
> ))
> Note, that the filesystem path is now:
> somedir/imagekitmedia/imagekit/media
> and the actaual URL pattern is now
> /media/imagekit/
> so if the img src now says
> django will not throw errors because that file will directly exist in:
> /somedir/imagekitmedia/media/imagekit/img/image.jpg
> I think option 2 is probably the simplest way to move forward..but it might
> require you to move some files around. The long term solution is that all
> the apps static directories should mirror how the templates/ directories
> function.
>
> Dan
> On Tue, Feb 2, 2010 at 4:33 PM, Nir Yariv wrote:
>>
>> In rapidsms.ini, media root is defined like so:
>>
>> media_root=file:///tmp
>>
>> Is there any Commcare specific code that relies on this? A grep didn't
>> come up with anything obvious. Django seems to be confused by this,
>> appending it to the project root and then throwing errors like:
>>
>> IOError: [Errno 2] No such file or directory: u'/Users/niryariv/
>> Projects/commcare-hq/file:/tmp/data/photos/hoover.jpg'
>>
>> The same goes for:
>>
>> cache_backend=file:///tmp/django_cache
>
>
>
> --
> Daniel Myung
> Senior Engineer, Dimagi, Inc
> 529 Main St, Suite 606 | Charlestown, MA 02129
> office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
> http://www.dimagi.com/
>
Yeah, i think that works out.
probably makes simplest sense to store it in duplicate in its own
directory...we can deal with the space implications later (famous last
words...heh)
but yeah, if nothing explodes when doing file manipulation then the whole
http path and local file path issue probably was resolved with the solution
you have.
Dan
Hi Daniel,
After posting the message to the list yesterday I kept working on
this, and came up with the following solution:
media_root is commented out in local.ini (or it could be left in
with an empty value)In apps/photos/urls.py I added the following mapping:
(r'^data/photos/(?P.*)$', 'django.views.static.serve',
{"document_root": 'data/photos/'}),
(Photos are stored under data/photos, next to the data/attachments etc
directories)What do you think? It seems to work ok, and doesn't break anything
(though I might well be missing something..). Storing under data seems
to make sense, since that's where data from the xforms gets stored. If
you think we can go ahead with this I'll move 'data/photos' to an
export_path setting in local.ini, as in receiver/xformmanager.Thanks,
NirOn Wed, Feb 3, 2010 at 9:57 AM, Daniel Myung dmyung@dimagi.com wrote:
hey nir,
so...the cache_backend is for file:///tmp/ is fairly standard django
practice. that ought to just autogenerate the right files...if you need
to
really makei t work, just make sure those paths exist and your user
executing them has the correct RW permissions.
regarding the MEDIA_ROOT and MEDIA_URL, it's a rather crazy bit. but i
think we have a workaround for that.
Long explanation
So, quick answer is that thereis no commcare specific rule because the
way
the rapidsms bootstrapper work, it hasn't really come up because the
rapidsms webapp/urls.py walks through all the apps/ directories and looks
for a static/ directory to map using the built in django debug server.
now, in a regular app, you probably can get away with this, but with
ImageKit or any other django app that needs to deal with files this
becomes
problematic. Here's why.
when rapidsms maps a static directory (to server, say, css and javascript
and static images)
it'll map it like this:
/apps/myapp/static/css/somestyle.css
/apps/myapp/static/img/image.jpg
built into the MEDIA_ROOT of django is the implicit /
so, when you see the image or stylesheet, it won't use the full hostname,
it'll just do relative pathing
so, <img src="/static/myapp/img/image.jpg>, say, instead of
Anyway, the issue here is notice how the paths flip from myapp/static/ to
static/myapp?
I'm guessing you're getting a bunch of errors with regard to Suspicious
Operation deep in the django file management modules. This is because
the
django file manager checks the MEDIA_ROOT with the end paths of the media
URL and the actual file url.
We have 2 suggested workarounds:
1: when we serve our stuff up directly in apache, we symlink all the
static/ directories to a single staticmedia/ folder in a separate
location.
we set the root of that folder to MEDIA_ROOT and the file suspicious
operations go away, as the path flipping no longer happens.
To see this in action look at setapache_static.py and also look for the
string USE_DJANGO_STATIC_SERVER to see how this works.2: For ImageKit in particular, you can also manually add its own path to
the urls to serve it up in a directory structure that the django system
will
like.
settings.py:
AUX_MEDIA_DIRS = {
'imagekit':
os.path.join(filepath,'somedir','imagekitmedia','media','imagekit'),}urls.py modifications
if hasattr(settings,'AUX_MEDIA_DIRS'):
if hasattr(settings,'USE_DJANGO_STATIC_SERVER') and
settings.USE_DJANGO_STATIC_SERVER:
for dirname, path in settings.AUX_MEDIA_DIRS.items():
if os.path.exists(path):
urlpatterns += patterns("", url("^media/%s/(?P.*)$"
%
dirname,
"django.views.static.serve",
{"document_root":
path }
))
Note, that the filesystem path is now:
somedir/imagekitmedia/imagekit/media
and the actaual URL pattern is now
/media/imagekit/
so if the img src now says
django will not throw errors because that file will directly exist in:
/somedir/imagekitmedia/media/imagekit/img/image.jpg
I think option 2 is probably the simplest way to move forward..but it
might
require you to move some files around. The long term solution is that
all
the apps static directories should mirror how the templates/ directories
function.Dan
On Tue, Feb 2, 2010 at 4:33 PM, Nir Yariv niryariv@gmail.com wrote:In rapidsms.ini, media root is defined like so:
media_root=file:///tmp
Is there any Commcare specific code that relies on this? A grep didn't
come up with anything obvious. Django seems to be confused by this,
appending it to the project root and then throwing errors like:IOError: [Errno 2] No such file or directory: u'/Users/niryariv/
Projects/commcare-hq/file:/tmp/data/photos/hoover.jpg'The same goes for:
cache_backend=file:///tmp/django_cache
--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
http://www.dimagi.com/
--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
Can someone clarify the duplicate storage? why are we storing it twice?
-J
Yeah, i think that works out.
probably makes simplest sense to store it in duplicate in its own
directory...we can deal with the space implications later (famous last
words...heh)but yeah, if nothing explodes when doing file manipulation then the whole
http path and local file path issue probably was resolved with the solution
you have.Dan
On Wed, Feb 3, 2010 at 10:29 AM, Nir Yariv niryariv@gmail.com wrote:
Hi Daniel,
After posting the message to the list yesterday I kept working on
this, and came up with the following solution:
media_root is commented out in local.ini (or it could be left in
with an empty value)In apps/photos/urls.py I added the following mapping:
(r'^data/photos/(?P.*)$', 'django.views.static.serve',
{"document_root": 'data/photos/'}),
(Photos are stored under data/photos, next to the data/attachments etc
directories)What do you think? It seems to work ok, and doesn't break anything
(though I might well be missing something..). Storing under data seems
to make sense, since that's where data from the xforms gets stored. If
you think we can go ahead with this I'll move 'data/photos' to an
export_path setting in local.ini, as in receiver/xformmanager.Thanks,
NirOn Wed, Feb 3, 2010 at 9:57 AM, Daniel Myung dmyung@dimagi.com wrote:
hey nir,
so...the cache_backend is for file:///tmp/ is fairly standard django
practice. that ought to just autogenerate the right files...if you need
to
really makei t work, just make sure those paths exist and your user
executing them has the correct RW permissions.
regarding the MEDIA_ROOT and MEDIA_URL, it's a rather crazy bit. but i
think we have a workaround for that.
Long explanation
So, quick answer is that thereis no commcare specific rule because the
way
the rapidsms bootstrapper work, it hasn't really come up because the
rapidsms webapp/urls.py walks through all the apps/ directories and
looks
for a static/ directory to map using the built in django debug server.
now, in a regular app, you probably can get away with this, but with
ImageKit or any other django app that needs to deal with files this
becomes
problematic. Here's why.
when rapidsms maps a static directory (to server, say, css and
javascript
and static images)
it'll map it like this:
/apps/myapp/static/css/somestyle.css
/apps/myapp/static/img/image.jpg
built into the MEDIA_ROOT of django is the implicit /
so, when you see the image or stylesheet, it won't use the full
hostname,
it'll just do relative pathing
so, <img src="/static/myapp/img/image.jpg>, say, instead of
Anyway, the issue here is notice how the paths flip from myapp/static/
to
static/myapp?
I'm guessing you're getting a bunch of errors with regard to Suspicious
Operation deep in the django file management modules. This is because
the
django file manager checks the MEDIA_ROOT with the end paths of the
media
URL and the actual file url.
We have 2 suggested workarounds:
1: when we serve our stuff up directly in apache, we symlink all the
static/ directories to a single staticmedia/ folder in a separate
location.
we set the root of that folder to MEDIA_ROOT and the file suspicious
operations go away, as the path flipping no longer happens.
To see this in action look at setapache_static.py and also look for the
string USE_DJANGO_STATIC_SERVER to see how this works.2: For ImageKit in particular, you can also manually add its own path
to
the urls to serve it up in a directory structure that the django system
will
like.
settings.py:
AUX_MEDIA_DIRS = {
'imagekit':
os.path.join(filepath,'somedir','imagekitmedia','media','imagekit'),}urls.py modifications
if hasattr(settings,'AUX_MEDIA_DIRS'):
if hasattr(settings,'USE_DJANGO_STATIC_SERVER') and
settings.USE_DJANGO_STATIC_SERVER:
for dirname, path in settings.AUX_MEDIA_DIRS.items():
if os.path.exists(path):
urlpatterns += patterns("",
url("^media/%s/(?P.*)$" %
dirname,
"django.views.static.serve",
{"document_root":
path }
))
Note, that the filesystem path is now:
somedir/imagekitmedia/imagekit/media
and the actaual URL pattern is now
/media/imagekit/
so if the img src now says
django will not throw errors because that file will directly exist in:
/somedir/imagekitmedia/media/imagekit/img/image.jpg
I think option 2 is probably the simplest way to move forward..but it
might
require you to move some files around. The long term solution is that
all
the apps static directories should mirror how the templates/ directories
function.Dan
On Tue, Feb 2, 2010 at 4:33 PM, Nir Yariv niryariv@gmail.com wrote:In rapidsms.ini, media root is defined like so:
media_root=file:///tmp
Is there any Commcare specific code that relies on this? A grep didn't
come up with anything obvious. Django seems to be confused by this,
appending it to the project root and then throwing errors like:IOError: [Errno 2] No such file or directory: u'/Users/niryariv/
Projects/commcare-hq/file:/tmp/data/photos/hoover.jpg'The same goes for:
cache_backend=file:///tmp/django_cache
--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
http://www.dimagi.com/--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
http://www.dimagi.com/
Not sure I understood - is it the thumbnails, or the attachment photos
being copied?
>>> > Anyway, the issue here is notice how the paths flip from myapp/static/
>>> > to
>>> > static/myapp?
>>> > I'm guessing you're getting a bunch of errors with regard to Suspicious
>>> > Operation deep in the django file management modules. This is because
>>> > the
>>> > django file manager checks the MEDIA_ROOT with the end paths of the
>>> > media
>>> > URL and the actual file url.
>>> > We have 2 suggested workarounds:
>>> > 1: when we serve our stuff up directly in apache, we symlink all the
>>> > static/ directories to a single staticmedia/ folder in a separate
>>> > location.
>>> > we set the root of that folder to MEDIA_ROOT and the file suspicious
>>> > operations go away, as the path flipping no longer happens.
>>> > To see this in action look at setapache_static.py and also look for the
>>> > string USE_DJANGO_STATIC_SERVER to see how this works.
>>> >
>>> > 2: For ImageKit in particular, you can also manually add its own path
>>> > to
>>> > the urls to serve it up in a directory structure that the django system
>>> > will
>>> > like.
>>> > settings.py:
>>> > AUX_MEDIA_DIRS = {
>>> > 'imagekit':
>>> > os.path.join(filepath,'somedir','imagekitmedia','media','imagekit'),
>>> >
>>> > }
>>> > urls.py modifications
>>> > if hasattr(settings,'AUX_MEDIA_DIRS'):
>>> > if hasattr(settings,'USE_DJANGO_STATIC_SERVER') and
>>> > settings.USE_DJANGO_STATIC_SERVER:
>>> > for dirname, path in settings.AUX_MEDIA_DIRS.items():
>>> > if os.path.exists(path):
>>> > urlpatterns += patterns("",
>>> > url("^media/%s/(?P.*)$" %
>>> > dirname,
>>> > "django.views.static.serve",
>>> > {"document_root":
>>> > path }
>>> > ))
>>> > Note, that the filesystem path is now:
>>> > somedir/imagekitmedia/imagekit/media
>>> > and the actaual URL pattern is now
>>> > /media/imagekit/
>>> > so if the img src now says
>>> > django will not throw errors because that file will directly exist in:
>>> > /somedir/imagekitmedia/media/imagekit/img/image.jpg
>>> > I think option 2 is probably the simplest way to move forward..but it
>>> > might
>>> > require you to move some files around. The long term solution is that
>>> > all
>>> > the apps static directories should mirror how the templates/
>>> > directories
>>> > function.
>>> >
>>> > Dan
>>> > On Tue, Feb 2, 2010 at 4:33 PM, Nir Yariv wrote:
>>> >>
>>> >> In rapidsms.ini, media root is defined like so:
>>> >>
>>> >> media_root=file:///tmp
>>> >>
>>> >> Is there any Commcare specific code that relies on this? A grep didn't
>>> >> come up with anything obvious. Django seems to be confused by this,
>>> >> appending it to the project root and then throwing errors like:
>>> >>
>>> >> IOError: [Errno 2] No such file or directory: u'/Users/niryariv/
>>> >> Projects/commcare-hq/file:/tmp/data/photos/hoover.jpg'
>>> >>
>>> >> The same goes for:
>>> >>
>>> >> cache_backend=file:///tmp/django_cache
>>> >
>>> >
>>> >
>>> > --
>>> > Daniel Myung
>>> > Senior Engineer, Dimagi, Inc
>>> > 529 Main St, Suite 606 | Charlestown, MA 02129
>>> > office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
>>> > http://www.dimagi.com/
>>> >
>>
>>
>>
>> --
>> Daniel Myung
>> Senior Engineer, Dimagi, Inc
>> 529 Main St, Suite 606 | Charlestown, MA 02129
>> office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
>> http://www.dimagi.com/
>
>
right, my irresponsible suggestion was the entire attachment photo being
copied to the ImageKit's own personal direcotry, which the thumbnails be
copied over.
I'm assuming the worst for imagekit and how new models are instantiated.
Ideally i think the structure should be:
image = Image(filepath='/data/attachment/image0001.jpg')
where that file is the actually uploaded image.
/data/attachment/image0001.jpg
/sized/image0001_small.jpg
/sized/image0001_thumb.jpg
does that seem possible?
Not sure I understood - is it the thumbnails, or the attachment photos
being copied?On Wed, Feb 3, 2010 at 11:48 AM, Jonathan Jackson jjackson@dimagi.com wrote:
Can someone clarify the duplicate storage? why are we storing it twice?
-JOn Wed, Feb 3, 2010 at 10:32 AM, Daniel Myung dmyung@dimagi.com wrote:
Yeah, i think that works out.
probably makes simplest sense to store it in duplicate in its own
directory...we can deal with the space implications later (famous last
words...heh)
but yeah, if nothing explodes when doing file manipulation then the
whole
http path and local file path issue probably was resolved with the
solution
you have.Dan
On Wed, Feb 3, 2010 at 10:29 AM, Nir Yariv niryariv@gmail.com wrote:Hi Daniel,
After posting the message to the list yesterday I kept working on
this, and came up with the following solution:
media_root is commented out in local.ini (or it could be left in
with an empty value)In apps/photos/urls.py I added the following mapping:
(r'^data/photos/(?P.*)$', 'django.views.static.serve',
{"document_root": 'data/photos/'}),
(Photos are stored under data/photos, next to the data/attachments etc
directories)What do you think? It seems to work ok, and doesn't break anything
(though I might well be missing something..). Storing under data seems
to make sense, since that's where data from the xforms gets stored. If
you think we can go ahead with this I'll move 'data/photos' to an
export_path setting in local.ini, as in receiver/xformmanager.Thanks,
NirOn Wed, Feb 3, 2010 at 9:57 AM, Daniel Myung dmyung@dimagi.com wrote:
hey nir,
so...the cache_backend is for file:///tmp/ is fairly standard django
practice. that ought to just autogenerate the right files...if you
need to
really makei t work, just make sure those paths exist and your user
executing them has the correct RW permissions.
regarding the MEDIA_ROOT and MEDIA_URL, it's a rather crazy bit. but
i
think we have a workaround for that.
Long explanation
So, quick answer is that thereis no commcare specific rule because
the
way
the rapidsms bootstrapper work, it hasn't really come up because the
rapidsms webapp/urls.py walks through all the apps/ directories and
looks
for a static/ directory to map using the built in django debug
server.
now, in a regular app, you probably can get away with this, but with
ImageKit or any other django app that needs to deal with files this
becomes
problematic. Here's why.
when rapidsms maps a static directory (to server, say, css and
javascript
and static images)
it'll map it like this:
/apps/myapp/static/css/somestyle.css
/apps/myapp/static/img/image.jpg
built into the MEDIA_ROOT of django is the implicit /
so, when you see the image or stylesheet, it won't use the full
hostname,
it'll just do relative pathing
so, <img src="/static/myapp/img/image.jpg>, say, instead of
Anyway, the issue here is notice how the paths flip from
myapp/static/
to
static/myapp?
I'm guessing you're getting a bunch of errors with regard to
Suspicious
Operation deep in the django file management modules. This is
because
the
django file manager checks the MEDIA_ROOT with the end paths of the
media
URL and the actual file url.
We have 2 suggested workarounds:
1: when we serve our stuff up directly in apache, we symlink all the
static/ directories to a single staticmedia/ folder in a separate
location.
we set the root of that folder to MEDIA_ROOT and the file suspicious
operations go away, as the path flipping no longer happens.
To see this in action look at setapache_static.py and also look for
the
string USE_DJANGO_STATIC_SERVER to see how this works.2: For ImageKit in particular, you can also manually add its own
path
to
the urls to serve it up in a directory structure that the django
system
will
like.
settings.py:
AUX_MEDIA_DIRS = {
'imagekit':
os.path.join(filepath,'somedir','imagekitmedia','media','imagekit'),}urls.py modifications
if hasattr(settings,'AUX_MEDIA_DIRS'):
if hasattr(settings,'USE_DJANGO_STATIC_SERVER') and
settings.USE_DJANGO_STATIC_SERVER:
for dirname, path in settings.AUX_MEDIA_DIRS.items():
if os.path.exists(path):
urlpatterns += patterns("",
url("^media/%s/(?P.*)$" %
dirname,
"django.views.static.serve",
{"document_root":
path }
))
Note, that the filesystem path is now:
somedir/imagekitmedia/imagekit/media
and the actaual URL pattern is now
/media/imagekit/
so if the img src now says
django will not throw errors because that file will directly exist
in:
/somedir/imagekitmedia/media/imagekit/img/image.jpg
I think option 2 is probably the simplest way to move forward..but it
might
require you to move some files around. The long term solution is
that
all
the apps static directories should mirror how the templates/
directories
function.Dan
On Tue, Feb 2, 2010 at 4:33 PM, Nir Yariv niryariv@gmail.com wrote:In rapidsms.ini, media root is defined like so:
media_root=file:///tmp
Is there any Commcare specific code that relies on this? A grep
didn't
come up with anything obvious. Django seems to be confused by this,
appending it to the project root and then throwing errors like:IOError: [Errno 2] No such file or directory: u'/Users/niryariv/
Projects/commcare-hq/file:/tmp/data/photos/hoover.jpg'The same goes for:
cache_backend=file:///tmp/django_cache
--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617)
580-3103
http://www.dimagi.com/--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
http://www.dimagi.com/
--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
i think it should be possible, but wouldn't it make more sense to keep
all images in the same dir? i don't know how many images are expected
to be there in total - is it in the dozens/hundreds/thousands...
when a form data is saved to /data, is it meant to stay there or is
the data then processed into the DB?
btw, is there anything ImageKit will need to do other than resize? if
it's just a matter of thumbnails, we might just enforce an image
width/height via css (possibly displaying a full size preview on
hover/click/etc) and then keep only the original image file.
>> >>> > Anyway, the issue here is notice how the paths flip from
>> >>> > myapp/static/
>> >>> > to
>> >>> > static/myapp?
>> >>> > I'm guessing you're getting a bunch of errors with regard to
>> >>> > Suspicious
>> >>> > Operation deep in the django file management modules. This is
>> >>> > because
>> >>> > the
>> >>> > django file manager checks the MEDIA_ROOT with the end paths of the
>> >>> > media
>> >>> > URL and the actual file url.
>> >>> > We have 2 suggested workarounds:
>> >>> > 1: when we serve our stuff up directly in apache, we symlink all
>> >>> > the
>> >>> > static/ directories to a single staticmedia/ folder in a separate
>> >>> > location.
>> >>> > we set the root of that folder to MEDIA_ROOT and the file
>> >>> > suspicious
>> >>> > operations go away, as the path flipping no longer happens.
>> >>> > To see this in action look at setapache_static.py and also look for
>> >>> > the
>> >>> > string USE_DJANGO_STATIC_SERVER to see how this works.
>> >>> >
>> >>> > 2: For ImageKit in particular, you can also manually add its own
>> >>> > path
>> >>> > to
>> >>> > the urls to serve it up in a directory structure that the django
>> >>> > system
>> >>> > will
>> >>> > like.
>> >>> > settings.py:
>> >>> > AUX_MEDIA_DIRS = {
>> >>> > 'imagekit':
>> >>> > os.path.join(filepath,'somedir','imagekitmedia','media','imagekit'),
>> >>> >
>> >>> > }
>> >>> > urls.py modifications
>> >>> > if hasattr(settings,'AUX_MEDIA_DIRS'):
>> >>> > if hasattr(settings,'USE_DJANGO_STATIC_SERVER') and
>> >>> > settings.USE_DJANGO_STATIC_SERVER:
>> >>> > for dirname, path in settings.AUX_MEDIA_DIRS.items():
>> >>> > if os.path.exists(path):
>> >>> > urlpatterns += patterns("",
>> >>> > url("^media/%s/(?P.*)$" %
>> >>> > dirname,
>> >>> > "django.views.static.serve",
>> >>> > {"document_root":
>> >>> > path }
>> >>> > ))
>> >>> > Note, that the filesystem path is now:
>> >>> > somedir/imagekitmedia/imagekit/media
>> >>> > and the actaual URL pattern is now
>> >>> > /media/imagekit/
>> >>> > so if the img src now says
>> >>> > django will not throw errors because that file will directly exist
>> >>> > in:
>> >>> > /somedir/imagekitmedia/media/imagekit/img/image.jpg
>> >>> > I think option 2 is probably the simplest way to move forward..but
>> >>> > it
>> >>> > might
>> >>> > require you to move some files around. The long term solution is
>> >>> > that
>> >>> > all
>> >>> > the apps static directories should mirror how the templates/
>> >>> > directories
>> >>> > function.
>> >>> >
>> >>> > Dan
>> >>> > On Tue, Feb 2, 2010 at 4:33 PM, Nir Yariv wrote:
>> >>> >>
>> >>> >> In rapidsms.ini, media root is defined like so:
>> >>> >>
>> >>> >> media_root=file:///tmp
>> >>> >>
>> >>> >> Is there any Commcare specific code that relies on this? A grep
>> >>> >> didn't
>> >>> >> come up with anything obvious. Django seems to be confused by this,
>> >>> >> appending it to the project root and then throwing errors like:
>> >>> >>
>> >>> >> IOError: [Errno 2] No such file or directory: u'/Users/niryariv/
>> >>> >> Projects/commcare-hq/file:/tmp/data/photos/hoover.jpg'
>> >>> >>
>> >>> >> The same goes for:
>> >>> >>
>> >>> >> cache_backend=file:///tmp/django_cache
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Daniel Myung
>> >>> > Senior Engineer, Dimagi, Inc
>> >>> > 529 Main St, Suite 606 | Charlestown, MA 02129
>> >>> > office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617)
>> >>> > 580-3103
>> >>> > http://www.dimagi.com/
>> >>> >
>> >>
>> >>
>> >>
>> >> --
>> >> Daniel Myung
>> >> Senior Engineer, Dimagi, Inc
>> >> 529 Main St, Suite 606 | Charlestown, MA 02129
>> >> office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
>> >> http://www.dimagi.com/
>> >
>> >
>
>
>
> --
> Daniel Myung
> Senior Engineer, Dimagi, Inc
> 529 Main St, Suite 606 | Charlestown, MA 02129
> office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
> http://www.dimagi.com/
>
i think it should be possible, but wouldn't it make more sense to keep
all images in the same dir? i don't know how many images are expected
to be there in total - is it in the dozens/hundreds/thousands...
I would think having them all in the same directory definitely makes the
most sense.
when a form data is saved to /data, is it meant to stay there or is
the data then processed into the DB?
It depends on what is submitted, but in general most of the forms are
processed into the database as well.
btw, is there anything ImageKit will need to do other than resize? if
it's just a matter of thumbnails, we might just enforce an image
width/height via css (possibly displaying a full size preview on
hover/click/etc) and then keep only the original image file.
Many of our deployments are in severely bandwidth-limited environments, so I
think we do need to serve up the thumbnails in a way that's doesn't require
sending the whole image over the wire.
thanks,
Cory
I would think having them all in the same directory definitely makes the
most sense.
sounds good. should the images in data/attachments be kept there or
can they be deleted once ImageKit imports them?
Many of our deployments are in severely bandwidth-limited environments, so I
think we do need to serve up the thumbnails in a way that's doesn't require
sending the whole image over the wire.
got it. is there a particular thumbnail size you have in mind?
I would think having them all in the same directory definitely makes the
most sense.sounds good. should the images in data/attachments be kept there or
can they be deleted once ImageKit imports them?
My sense is that for now we leave them and solve that problem later.
Deleting them is dangerous because any code that doesn't know you're
treating images differently will expect them there. Not sure if anyone has
a better plan though.
Many of our deployments are in severely bandwidth-limited environments,
so I
think we do need to serve up the thumbnails in a way that's doesn't
require
sending the whole image over the wire.got it. is there a particular thumbnail size you have in mind?
Dan, what size were the images on the last iteration using?
for zambia we were doing thumbs at like 120 or 150 pixels in width
a standard display size was 640x480 to 800x600. for MOST situations that
size was actually sufficient to make a diagnosis and never have to view the
full sized image.
On Wed, Feb 3, 2010 at 2:12 PM, Nir Yariv niryariv@gmail.com wrote:
I would think having them all in the same directory definitely makes the
most sense.sounds good. should the images in data/attachments be kept there or
can they be deleted once ImageKit imports them?My sense is that for now we leave them and solve that problem later.
Deleting them is dangerous because any code that doesn't know you're
treating images differently will expect them there. Not sure if anyone has
a better plan though.Many of our deployments are in severely bandwidth-limited environments,
so I
think we do need to serve up the thumbnails in a way that's doesn't
require
sending the whole image over the wire.got it. is there a particular thumbnail size you have in mind?
Dan, what size were the images on the last iteration using?
--
Daniel Myung
Senior Engineer, Dimagi, Inc
529 Main St, Suite 606 | Charlestown, MA 02129
office: (617) 580-3100 | mobile: (617) 863-0678| fax: (617) 580-3103
so, 3 sizes: 120 width, 640 and original? (I suggest specifying only
width so ImageKit will scale to the appropriate height automatically,
retaining the image proportions)