Tuesday, June 09, 2009

Who says Application Express can't scale?

Over a year ago, I wrote about who is using Oracle Application Express on http://apex.oracle.com. At the time, I included some weekly statistics.

During our team meetings every week, I report on the overall usage of http://apex.oracle.com. I just wanted to report on some astounding numbers:


Total Page Views Distinct Applications Distinct Users
---------------- --------------------- --------------
6225566 3268 2778


Total Workspaces
----------------
11643


Total Applications
------------------
39849


Workspaces Approved
-------------------
316

That's 6,225,566 page views in the past week, and this is still running on the same Dell PowerEdge 1950.

Now granted - the ability to sustain this type of load throughout the week deserves some credit to Kris Rice, who spent a fair amount of time analyzing Application Express on apex.oracle.com and identifying some major problem areas in performance. As well, he brought in some experts to perform OS tuning. The recent purge on apex.oracle.com removed close to 10,000 workspaces, schemas, tablespaces and datafiles. Lastly, a couple months ago, I implemented a Resource Manager plan which prevents someone from monopolizing the entire server. I'll be presenting a detailed discussion of Resource Manager and apex.oracle.com at ODTUG Kaleidoscope 2009.

A large number of these page views are from the bots of Search Engines, especially since ProMED Mail is run on apex.oracle.com. But a page view and execution of the engine is a page view and execution of the engine. All in all, that's a large load on a relatively cheap piece of hardware.

5 comments:

Jornica said...

The login screen of apex.oracle.com says: "This service is for demonstration purposes only. Use of this system to store sensitive data is prohibited". Is the ProMED-Mail application for demonstration purposes only?

I always told my colleagues to use apex.oracle.com as a "playground"
and not to use it for production purposes.

With kind regards,

Jornica

Joel R. Kallman said...

Hi Jornica,

That is a true statement and the advice you told your colleagues is still accurate. ProMED-Mail is actually run by us (Oracle), and we just decided to consolidate it onto this server.

I intend to consolidate AskTom to apex.oracle.com too - so that will be a "production" application, too. But if you're not AskTom and you're not ProMED-Mail, then use of apex.oracle.com for anything other than demonstration purposes is prohibited.

Joel

Scott said...

I have a table with ~2500 Bot IP addresses - let me know if you want it, as you can remove the "chatter" from the actual page views.

- Scott -

mroessler said...

Joe,

Perhaps you have shared this elsewhere - but if not, it would be useful to learn about the specific os-related improvements you mentioned that were made on apex.oracle.com in order to boost apex application performance.

Have you shared anywhere the specifications on the Dell 1950 running apex.oracle.com - such as RAM and IO?

Thank you,

Michael Roessler

Joel R. Kallman said...

@Michael,

In the comments on the original post, I gave some of these details. Specifically:

Memory - 16GB 667MHz (8x2GB), Dual Ranked DIMMs"

As far as the I/O, I don't think it's anything special. It is just hooked up to a big NetApp filer that Global IT manages.

"Joe"