Looks like Eclipse PDT was getting quite slow on me, it thinks sometimes 10-15 seconds when I switch between files or save one. My current project is not small one, but still…
Looking around for a solutions I found quite easy tip at
http://www.eclipsezone.com/eclipse/forums/t61618.html
In short – I am now running
eclipse.exe -vmargs -Xms256m -Xmx256m -XX:PermSize=64m -XX:MaxPermSize=64m
and my Eclipse is way faster.
Thanks for the tip, Riyad Kalla!
One of the things I wish I had found in Google faster when trying to figure out why PHP script refuses to connect to remote MySQL server issuing an error
SQLSTATE[HY000] [2003] Can't connect to MySQL server on 'xxx.xxxx.xxx.xxx' (13)
I am able to connect from local shell, so first thought was if something wrong with recent Zend Framework upgrade, but after a while I figured out that answer is very simple – SELinux was blocking remote connections from PHP scripts executed by Apache web server. The error code (13) at the end of error message means “permission denied”, so that’s the indication to see if you have similar issue or not.
Anyway, login as root and do
setsebool -P httpd_can_network_connect=1
to make it work.
Of course, think twice because you make web server a bit less secure, so don’t do that unless you are sure you need it.
I am using Joomla for the content management, and it looks like my site is getting slow, no, not just slow, it’s SLOW.
Quick check in the MySQL database showed that there is no really big tables which could cause some delays if not indexed properly or getting locked. So, next step would be naturally looking into the PHP code itself trying to identify bottlenecks.
First of all – copy everything to test site – I don’t really want to shut the site down or display some debug info to the visitors. My test site is actually a linux box which is pretty old one, so it probably will run slower comparing to production, but hopefully still allow me to find which areas of code are causing that.
Next step – tools. To profile PHP scripts I’ll use APD (http://pecl.php.net/package/apd). Best case scenario – you type
pecl install apd
in the shell to install package from PECL.
You may also need to add it to PHP configuration, I just created extra config file
/etc/php.d/apd.ini
with content
[apd]
zend_extension="/usr/lib/php/modules/apd.so"
apd.dumpdir = /var/apd
apd.statement_tracing = 0
(make sure directory specified in dumpdir is created and writable for PHP)
Now we need to add
apd_set_pprof_trace();
at the beginning of the index.php script and open the page we want to profile, front page would be a good place to start.
In the dump directory a file created after each page opening, something like
/var/apd/pprof.08711.0
Let’s see what consumes resources now, by running
/usr/bin/pprofp -R /var/apd/pprof.08711.0
Oh, that hurts:
Total Elapsed Time = 23.58
Total System Time = 0.65
Total User Time = 7.18
Log in to Joomla admin, go to Extensions->Module Manager, delete all modules I do not really use, but which came pre-setup with Joomla.
Total Elapsed Time = 19.51
Total System Time = 0.24
Total User Time = 2.48
Ok, I am starting to see results already 
Now let’s pay attention to the actual calls and time spent by script in functions.
14 seconds spent in JDocumentRendererModule->getFeed…
Hm… I do have a feed from my blog, now I wonder what the cache settings for that block is. So, in admin Module Manager -> mod_feed, advanced settings, cache : use global, time 60 minutes (I think I am ok with blog posts delayed for 1 hour, but it’s your call, just make sure it won’t come on every page load). That slashed a lot from the page load – over 15 sec of wasted time (note – check why blog feed is so slow, there should be some reasons!!!)
JDocumentHTML->_parseTemplate – I reviewed template and got rid of calls for modules which are not required or not used (like user1, user2, user3 areas which came with template and similar) – that saved 1-2 seconds per call.
At this point I have
Total Elapsed Time = 2.82
Total System Time = 0.21
Total User Time = 2.48
Which is way better, but still makes me wonder if it can run faster.
There are other options to play with
/usr/bin/pprofp -u /var/apd/pprof.09573.15
gives an idea what subroutines consumed most user time.
Last but not least: for the profiling purposes I turned off cache (Site->admin->global configuration). For life site it should be turned on with reasonable time to keep cached copy, that shaves off another 50% of execution time on subsequent calls of the same page.
Total Elapsed Time = 1.78
Total System Time = 0.17
Total User Time = 1.55
For now it’s as good as it gets on my old server, next thing to do would be profiling the script on the production server to see what it looks like there.
Another tip: if you profile a live site, restrict profiler calls to your own IP:
if($_SERVER['REMOTE_ADDR'] == "xxx.xxx.xxx.xxx") {
apd_set_pprof_trace();
}
otherwise you may get buried in a number of trace logs.
I just discovered interesting thing – to my shame I misspelled word “developer” (as in web developer) on my web site and for a while I was advertising myself as “web developmer“. Bad for me I guess. But interesting thing, when checking statistics, I get better position in the search results as a developMer – just because there is not a lot of pages with same typo. Just to compare – when I search for the web developer it returns over 56 millions of pages and for web developmer it’s only 4500. So, I guess if I add more words like this to my page saying that I do dtabase desing and php programing as well as software analitics, I have good chance to pop up higher in Google search results. The question is, however, do I really want people to see pages with those typos and if those people will be willing to hire guy with spelling like that and no attention to details. Well, I guess some people won’t notice, but probably I don’t really want to take a chance – so fixing the spelling and having a good laugh. At least I still have a blog entry with developmer in it!
Quite an interesting idea – to execute PHP scripts in Windows as regular exe. Behind the scenes it uses IE component, but there is no IE toolbars and you have a control on windows your application uses. Also you can create an installer, so it will look for user as a regular windows application.
These days a lot of applications are actually switching to web services, so you actually do not have to install software, you can just use it from your internet browser. But I can still see quite a few uses of this tool to create client site components for the application.
So, will keep this in mind and may be one day will use it not just to play.
Do you know PHP and thinking about getting some freelance job? You are not alone for sure, freelancing is a good way to get an extra income, to get started building your portfolio or to stop coming to the office and work from home.
Here is a good list of freelance positions for PHP developers:
http://workfromhomephpjobs.blogspot.com/
The biggest traffic source for the web site is search engines traffic. There is a lot what can be written on search engine optimization (SEO). But I just realized that often web masters or web site owners missing quite important application from the Google itself
That’s a valuable data on what Google “knows” about your web site, definetely worth checking and should be a starting point of the search engines optimization.
If you are doing web development, then you know that you need to have all recent versions of the IE browser to troubleshoot your sites. Here is one way to have them all, thanks to Edskes Software collection – just download pack and install all versions you want.
We are using FogBugz application for the bugs and support requests tracking. And it’s configured to receive support@companyname.com emails (means tons of spam). Once the database size hit approximately 20,000 requests, we started really wonder: why so slow? It looks like there was another table BugEvent which size is around 200,000 records at that time.
It could be some other application, could be your own code. Doesn’t matter – just it happens, one database size grows, performance of the application is degrading.
First step to troubleshoot – enable slow queries log in the MySQL config file (/etc/my.cnf is popular location). Open file in editor and add lines
log_slow_queries = /var/log/mysql-slow.log
long_query_time = 10(path for the log file is totally up to you, I use MySQL 5.0*, so in other versions syntax can be different)
Those lines tell mysql to log all queries lasted 10 seconds or more. Restart MySQL now and wait.
After a while run mysqldumpslow as root. If there are any slow queries logged, you will see summary, here is mine:
Reading mysql slow query log from /var/log/mysql/mysql-slow.log
Count: 36 Time=549.81s (19793s) Lock=0.00s (0s) Rows=0.0 (0)
SELECT Bug.ixBug AS ix, Bug.ixBugEventLatest AS ixChild,
Area.nTypeAs nAreaType
FROM (Bug INNER JOIN Area ON Bug.ixArea = Area.ixArea)
WHERE ixBugEventLatest <= N
AND ixBugEventLatest >= N AND Bug.ixBug IN
(SELECT ix FROM IndexDelta WHERE sType = 'S'
AND fDeleted = N)
AND -N = -N
ORDER BY ixBugEventLatest DESC
LIMIT NQuery listed above appears to take around 10 minutes to execute. Now check the tables type in the MySQL
mysql> show table status;
In our scenario I found that all tables are MyISAM. That means, that while query is executed (10 minutes) tables used in it will be locked for writing. So, if someone tries to update bug, they have to wait up to 10 minutes.
Check what MySQL engines you have:
mysql> show engines;
+---------+----------+---------------------------
| Engine | Support | Comment
+---------+----------+---------------------------
| MyISAM | DEFAULT | Default engine as of MySQL
| MEMORY | YES | Hash based, stored in memory
| InnoDB | YES | Supports transactions, row-
...If InnoDB is supported, then all what has to be done to speed up application is switching to InnoDB in this situation. This will not make query above run any faster, but, that will not lock the whole table for the time query is running. There are some benefits of MyISAM table type (well, it’s faster!), so you may don’t want to convert all tables to InnoDB at this point. But large tables with concurrent updates and reads will benefit.
If InnoDB is not supported, it can be turned on in MySQL configuration file, just comment out skip-innodb line.
In case of FogBugz I found the following tables to be worst offenders:
Bug, BugEvent, BugRelation, SorterToken, TokenAssociation.
To convert those, use:
mysql> alter table Bug engine=InnoDB;
After conversion performance of the application increased immediately, as there are no more locks on popular tables.
There are additional benefits of InnoDB engine, such as foreign keys. Read more on InnoDB:
I’ve being working with the application where users can post their property listings and then upload unlimited number of photos for their property. As the web site has being around for quite a few years, previous solution was straight forward – form with multiple file input fields. As digital cameras started producing images with more and more megapixels, and larger images as a result, this method was getting worse and worse. When user uploads multiple photos, they have no way to tell what the progress is, so people sometimes were re-submitting more and more of the same images. We added pop-up with fake progress bar (animated image saying that upload in progress), but started looking for a better solution.
First find was Java applet which allows user to select multiple files from their hard drive and then uploads those files with FTP. After upload is completed, user is redirected to the web page which does the processing of uploaded files. This solution appeared to be quite good and worked fine for a couple of years. Well, when user opens page, they are warned that there is Java applet which may be a security/privacy issue, so they have to confirm that applet is Ok. There were also reports that it was not working on Macs, which may be issue of their browser/java configuration, but you probably still want users to be able to use your page with no extra tweaking on their computers.
Finally, the age of AJAX is here, and most “big” web sites are using some fancy uploaders for the files (think Gmail attachments, etc), which makes us wonder if we should join the crowd. My quick research brought Dojo fileUploader widget to the spotlight. I worked with Dojo framework before, so it sounded like a good idea. The idea is simple – there is Flash component which does files selection and upload, and JavaScript methods which are called on events like File Selected, Upload Progress, Upload Complete, so you can report progress back to user, execute action when upload is complete, etc.
It appeared quite easy to setup uploader, I started with stable release of Dojo 1.2 and in a bit of time had working solution.
Unfortunately then the problem started. First of all, this solution seems to be quite fragile – as it was tested on different browsers it started getting broken one way or another. We tried to upgrade Dojo libraries to 1.3 beta 2 and then 3 releases, which appeared to work better, but still was very sensitive. For example, for computers with older versions of Flash it was causing issues throwing JavaScript errors, overfilling stack and memory. Another problem seems to be with the upload itself – when uploading many files at once, it appeared to use a lot of resources on the users computer and some users were reporting crashed browsers. I tried to play with versions 1.2 and 1.3 (detecting flash version, for example, and providing Flash 9 users with version 1.2 of Dojo, and Flash 10 users with 1.3), but it still was quite unreliable.
After spending lot of time with troubleshooting, I looked at alternative solution from Yahoo User Interface Library, version 2.7.0. Their Uploader widget uses the same idea behind the scenes, but, after installation and minor troubleshooting (won’t start working immediately, was doing nothing first – same when I ran example from their official web site) it appeared to be quite stable, working in all browsers we tried (XP, Vista, Mac OS, Firefox, IE 6 and 7, Flash 9 and 10). And most of all – they have variable to limit number of simultaneous uploads, which allows you to limit resources usage. User interface is nicer out of the box (sure you can do same thing with Dojo, but not 
when you are lazy and want quick solution…)
So, for the time being, if someone asks me how to implement multiple file uploads with progress bars – my answer would be YUI uploader.