The SugarCRM cache directory demystified

by Sander Marechal

Updated 2009-01-07. One of the projects I am working on at the moment at my day job is customizing SugarCRM. We want to start managing all changes to SugarCRM using subversion, so we need to know which bits should be under version control and which bits should not. During my analysis I came across the ill‐named and infamous cache directory.

SugarCRM’s cache is anything but that. I already knew that uploaded files and e-mails were also stored in the cache directory, but I needed to know exactly what was in the cache so I can make some informed decision on what to put under version control and what bits of a production system should be backed up. I found my answer, after a whole lot of searching, reading the SugarCRM source code and asking around. It turns out that the cache directory contains a hodgepodge of user data files, configuration files and—oh yes, there are some real cache files in there as well.

If there is one thing to take away from this article it’s “Don’t mess with the cache unless you know what you’re doing”. That bears repeating:

“Don’t mess with the cache unless you know what you’re doing”

With that firmly burned into the back of your eyeballs, here’s an overview what the cache directory contains.

cache/

This top‐level cache directory contains many subdirectories. All the standard directories are described below. Sometimes Sugar creates even more directories in here, for example when upgrading from one version to the next. Unless explicitly noted below you should never delete any of the files under the cache directory.

cache/blowfish/

This directory stores the blowfish encryption keys that SugarCRM uses to encrypt and decrypt your POP3, SMTP and IMAP passwords. If you remove them, SugarCRM can’t decrypt the passwords anymore and you will need to re‐enter them.

cache/csv/

Exports in CSV format are stored here temporarily before you download them. You can safely remove these.

cache/dashlets/

Sugar keeps a list here of all dashlets that you can use. You can safely remove it. Sugar will recreate it automatically when it’s missing.

cache/diagnostic/

When you run Sugar’s diagnostic tool, this is where it stores the diagnostics file. You can safely delete these files.

cache/dynamic_fields/

This directory contains caches of all the custom modules and fields in order to speed up Sugar. You can safely delete it. When data is missing from it, Sugar will recreate it automatically. If you use the “Quick Repair and Rebuild” function it will also be cleaned and repaired.

cache/feeds/

When you users subscribe to RSS feeds in SugarCRM, they will be downloaded to this directory before being displayed to the user. You can safely delete these files. Sugar will re‐download them.

cache/generated_forms/

If you have created any web-to-lead capture forums, this is the place where they get stored. If you use these forms directly on your website (for example via an i‐frame) then you cannot delete them. But if you are like me and simply used them to create the correct HTML for your own website then you can safely remove these forms.

cache/images/

This is a very nasty directory because it contains multiple types of data. Do not mess around with it!

The first type of data it contains are images that you embed or attach into outgoing mail. These images are cached here before actually being attached to the e‐mail when you hit “Send”. If you are using SugarCE (Community Edition) then you can safely remove these files.

If you are a SugarPro or Sugar Enterprise user then you’re out of luck. These editions of SugarCRM come with the Knowledge Base add‐on module. This directory is the permanent storage area of all the files and images you embed and attach to your Knowledge Base documents. You should not delete these!

This directory is a prime example of how not to use a cache. Bad developer, no Twinkie for you!

cache/import/

When you import data in csv format, the uploaded files will be stored here. You can safely remove them.

cache/jsLanguage/

Here you will find the JavaScript versions of all the language files. If they are missing they will be regenerated by the Sugar MVC View class, so can safely delete them if you want to.

cache/layout/

In Sugar 4.0.1 and below, this directory holds the HTML files for the views that you customized with Sugar Studio. In Sugar 4.5.x and up this directory doesn’t seem to be used any more because the custom views are now stored in the custom/ and modules/ directories. So, this directory should be empty, but if you run an old Sugar (4.0.1 or prior) then do not delete the contents of this directory.

cache/modules/

This is the directory where SugarCRM stores caches for all of the modules, templates, views, vardefs and more. You can safely remove most of these files, except some directories in the cache/modules/emails directory as listed below. They will be regenerated automatically when they are missing.

cache/modules/emails/*

Updated: Thanks to spaps for this one. If your users are using their e-mail inside Sugar then the emails cache directory will contain multiple directories with uuid directory names (like 1e1c2480-473f-ee19-16f0-48c63de4bec8). You must not delete these.

cache/pdf/

When you download a report in PDF format and Sugar cannot stream it directly to your browser, it will store the PDF file in this directory and offer it as a download. They can safely be removed.

cache/smarty/

This directory contains three subdirectories that hold caches and configuration files for the Smarty templating engine that is used by the new MVC system in Sugar 5.0 and up. For a detailed description of what they are for you can read the Smarty manual. You should not remove these directories.

cache/smarty/cache/

Smarty is able to cache static HTML version of templates which are stored here. Sugar does not use this functionality so this directory should be empty. If it’s not empty then you can safely clear it.

cache/smarty/configs/

Here live the configuration files for the Smarty templating engine. It should be empty (as of Sugar 5.1RC), but if a future Sugar update ever installs a file in this directory it should not be deleted because it’s a configuration file.

cache/smarty/templates_c

This directory contains compiled versions of all the Smarty templates. They are similar to the cached templates but they are dynamic PHP code instead of static HTML code. Smarty will automatically regenerate them if you delete them.

cache/upload/

This contains all the files that you uploaded or attached to things in Sugar as well as your installed modules and upgrades. The contents of this directory should not be deleted. If you do, you will loose all the e‐mails, attachments and other files that your users added to Sugar!

cache/xml/

The pipeline graph and various other graphs that SugarCRM displays are generated from XML files. These XML files are stored here. Sugar will regenerate them so you can safely delete the contents of this directory.

There is one directory that I haven’t quite figured out yet. On the SugarCRM forums a user called SugarDev.net posted a recipe for getting SugarCRM under Subversion control. He manually created the cache/data directory. None of my Sugar installations have that directory and grepping the SugarCRM source code yielded me no results about it either. When I do figure it out I will update this article accordingly.

Creative Commons Attribution-ShareAlike

Comments

#1 CRM Migration (http://www.mrvoip.com.au)

Nice info. I'm in the middle of a CRM Migration and need as much of the undocumented stuff as I can get my hands on to make an "informed" decision on things.

Looks like a bit of smarty template reading is on order too.

#2 Sander Marechal (http://www.jejik.com)

Smarty is actually pretty easy to use and well documented. I use it a lot and highly recommend it for other projects as well.

#3 spaps

Hi Sander!

Thanks for a comprehensive listing!
Much better than the answer I got from Sugar Support.

But I like to add some info about the cache/modules/emails/ directory:

If the SugarCRM users are using email in the system, you will get subfolders named something like "1e1c2480-473f-ee19-16f0-48c63de4bec8"
(it's the users id, there will be one for each user) under cache/modules/emails/.

Do not delete them or their content.

#4 Sander Marechal (http://www.jejik.com)

Thanks for you addition spaps. I have updated the article with your information.

#5 Vishwas

Thanx,
It will b useful for newcomer like me to understand sugar in detail.

#6 Ragin Hari


"cache/modules/
You can safely remove most of these files, except some directories in the cache/modules/emails."


Any files beyond modules/emails that cannot be removed?

#7 Sander Marechal (http://www.jejik.com)

Not that I know of, at least for Sugar 5.1. It's quite possible that Sugar 5.2 adds new cache files that aren't really cache files though.

#8 SugarDev.net (http://sugardev.net)

Hi Sander, I will look into why the ./cache/data is in my recipe. Perhaps it's a leftover from a long-gone version of Sugar.

Great work on the articles here by the way.

Loek

#9 Vince Synolia (http://www.synolia.com)

Hello,
thank a lot for this post, your article is really interesting and allow me to find one article for a lot of spread infos.
I did a french translation (not a pure translation by the way) for the french community (crm-france.com) and put a link back to your article, hope you don't mind ?
Here is the french version http://www.crm-france.com/dernieres-nouvelles/le-repertoire-cache-de-sugarcrm-demystifie.html

Regards,

Vince

#10 Sander Marechal (http://www.jejik.com)

I don't mind at all Vince :-) All my articles are licensed under a "Creative Commons 3.0 - Attribution - Share Alike" license.

#11 Rick Byers (http://www.ctigroup.com)

Is there any way to move the cache off the web server and onto another server (say the DM server) as I am not happy with all this data living in the DMZ!!

#12 Sander Marechal (http://www.jejik.com)

There are several ways, but none solve the problem with the DMZ, since the cache still has to be accessible by SugarCRM. You can move cache/ to a separate server and then mount it using samba or NFS on the correct directory. You could specify a different upload path in your configuration files and then move cache/upload/ to a networks share of some kind.

In both cases the data is on a different server but could potentially still be accessed through the DMZ server.

The only other thing I can think off is moving all of SugarCRM to a different server outside the DMZ and then use a proxy solution like Apache's mod_proxy to give access to SugarCRM from the outside.

#13 marsclark

Sander:
Very helpful information to me as I administer SugarCRM for our company.
My question is, can I apply the suggestions for removing files from the various cache subfolders to our 5.2 version? I ask because your post #7 makes me think this is for 5.1 and maybe not 5.2?

#14 Sander Marechal (http://www.jejik.com)

Hi Marsclark. You should be able to do that, but keep one thing in mind: Don't delete anything not listed above. It's possible that 5.2 adds even more directories under cache/ that I have not described above.

So, when in doubt: don't delete.

#15 treats

We want to start managing all changes to SugarCRM using subversion, so we need to know which bits should be under version control and which bits should not.

Did you ever conclude what needed to be under version control to allow you to drop your revisioned code onto a fresh install of sugar and get out your production version again?

#16 Sander Marechal (http://www.jejik.com)

@Treats: Yes. See my articles Keeping SugarCRM under Subversion control and Build custom SugarCRM modules in Subversion. Be sure to read the comments attached to those articles as well.

Also, here is a list of all my SugarCRM articles.

#17 Loek van Gool (http://sugardev.net)

Another year, another SugarCRM release and another sugardevtools update. I've mapped the 5.5 cache dir and have discovered six directories that are kept even while they could probably be deleted. My aim is to provide full cache deletion in the next release of sugardevtools, including SugarCache contents (that system can be quite irritating when developing language packs).

#18 Sander Marechal (http://www.jejik.com)

Thanks Loek. Do you happen to have a list of these six new directories? That ay I can update this article for those who want to do it themselves.

#19 Niklas (http://www.oribium.se/CRM)

Great article!
SugarCRM can be a jungle every now and then depending on what you're doing, and its work like this that helps people like me to find the right path out - thanks!

#20 Cesspool

Thanks! this info will be helpful to a newbie like me...hopefully all info still applies to ver 6.0+

#21 Mark (http://markvletter.nl)

cache/upload/upgrades also contains the updates you ran and modules you installed as a zip. These can be large and can also be deleted.

#22 Wyatt (http://whatan00b.com)

@Mark -- this directory can be deleted with latest versions, but is required by older versions of Sugar. I'm fairly certain it's safe to delete past 6.0, but I can't remember the exact version.

#23 Alex (http://www.mohuweb.com)

@Mark@Wyatt Under 5.5.4 cache/upload/upgrades/temp contains directories (named randomly) with uploaded files and can be safely removed.

#24 Keilaron (http://keilaron.ca/)

In some version of SugarCRM - I'm maintaining 6.2.4 and 6.1.3 right now, and I think 6.1.3 doesn't exhibit this behaviour - the cache/modules/* directories become a bit of a minefield. It's not that you need to keep them, it's that Sugar no longer stores the full metadata inside the Ext/Extension directories.. so I find myself having to copy (most of!) the metadata from the cache into the appropriate vardefs file when creating new fields, otherwise when updating staging/prod and doing a repair, I don't get the new fields.
Very annoying.

#25 Tony (http://www.amazon.com)

In every application I've ever seen a cache is used for storing data that can be recalculated if it is not there. If the app relies on data being there, it shouldn't be in the cache. Another example of poor design in SugarCRM.

#26 sugaroutfitters (https://www.sugaroutfitters.com)

If the app relies on data being there, it shouldn't be in the cache. Another example of poor design in SugarCRM.


This is simply a misunderstanding on how SugarCRM works and the best practices around managing multiple instances (dev/test/prod). Nothing is relied upon being in the cache. You can spin up a copy of an instance without the cache and it'll rebuild it appropriately.

#27 Keilaron (http://keilaron.ca/)

Wrong, sugaroutfitters. As I implied in my previous comment, create some fields via Studio, then delete your cache and you'll soon find yourself with missing or incorrect fields... the Extension directory is no longer being given all the data it is supposed to have.

It IS bad design. I've been going over how to keep multiple instances (because we DO maintain dev, test, and prod environments for all our clients) and I noticed that when uploading some changes for newer versions of Sugar, repair didn't repair; it expected some data that the dev copy had only placed in the cache.

#28 Karl (http://www.yellow.co.nz)

An important update on all this from SugarCRM is here: http://developers.sugarcrm.com/wordpress/2012/06/27/making-sugar-a-bit-more-filesystem-friendly-every-day/

A lot of the issues around things being in cache that shouldn't be have now been resolved - in particular the Blowfish Ciphers and Email module!
Post a new comment

Registration is not required to post comments, but cookies must be enabled. One of the advantages of registration is that you can edit your comments later on (editing not yet implemented). You can register or login here.




Your e-mail address will not be published, but your website URL will. All links that you post will tagged rel="nofollow" to throw off spammers. You are allowed to use the following XHTML tags in your comment: <em> <strong> <u> <b> <i> <strike> <blockquote> <big> <small> <ul> <ol> <li> <a href=""> <pre> <code> <tt> <br>. Please allow up to 60 second processing time after you post a comment. Our spam filters may take some time.