ETags and Action Caching

Added the action_cache plugin

    http://agilewebdevelopment.com/plugins/action_cache

which does action-caching with ETags support. The built-in Rails ETags "solution" sucks, because it forces a page-rerender, even when the content is unchanged.
This commit is contained in:
Jacques Distler 2007-05-25 22:52:42 -05:00
parent 6b21ac484f
commit d62b880e3f
11 changed files with 697 additions and 0 deletions

57
vendor/plugins/action_cache/CHANGELOG vendored Normal file
View file

@ -0,0 +1,57 @@
=== v0.0.1
* Initial implemenation
* Plugin implementation
=== v0.0.2
* Add the ability to replace the fragment_key method to do your own thing
=== v0.0.3
* Add timed expiry of action cache items with response.time_to_live = x
=== v.0.0.4
* Set the max-age value of the Cache-Control header to be the response.time_to_live
value if it is set, or 1 second if not
=== v.0.0.5
* Changed the Last-Modified header setting to not set Time.now if the header has already
been set. Fix from Eli Miller.
=== v.0.0.6
* Added encoding/decoding of the response body to allow UTF-8 encodings to not break due to
YAML bugs. Fix from Hui
* Handle potential problem with LastModified not being set correctly
* Add some simple tests using the Plugin Test Kit system
=== v.0.1.0
* Added support for the X-Sendfile feature in lighttpd
* Changed fragment usage to have body and headers in different fragments to allow X-Sendfile to work
* Refactored the code to make it easier to read
=== v.0.1.1
* Add the ability to decide whether to cache a request at request time
=== v.0.1.2
* Add support for the X-Accel-Redirect header in nginx
=== v.0.1.3
* Add the ability to enable the X-Sendfile feature by sending a HTTP_X_ENABLE_X_SENDFILE request
header for when lighttpd doesn't create the Rails process
=== v.0.1.4
* Change cache key generation from fragment_key member of an internal class to be an method
on the application controller that you can override in your own code.
* Make expire_action work with the user generated cache key
* Add an expire_all_actions method to clean everything out

20
vendor/plugins/action_cache/MIT-LICENSE vendored Normal file
View file

@ -0,0 +1,20 @@
Copyright (c) 2005-2006 Tom Fakes
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

174
vendor/plugins/action_cache/README vendored Normal file
View file

@ -0,0 +1,174 @@
Contributors:
Tom Fakes (tom@craz8.com) - Initial implementation, plugin implementation, x-sendfile work
Scott Laird - Ideas for the timed expiry and programmable fragment_key features
=== Action Cache Upgrade
This is a drop in replacement for the Rails Action Cache. When this plugin is
installed, the new behavior will take effect without any further configuration.
All documentation for the Rails Action Cache is still relevant. Sweepers still work, all the
fragment stores are supported.
See my blog at http://blog.craz8.com to find some interesting uses of the extended behavior
provided by this plugin
=== Major Change!
This version uses a different cache key generation mechanism. Instead of setting
ActionController::Caching::Actions::ActionCacheFilter.fragment_key, the cache code calls out to
the action_fragment_key method on the current controller. A default version of this method is
supplied that emulates the Rails built in Action Cache. If you haven't set the fragment_key
in your code, then nothing changes. If you have set the fragment_key, then you will need
to move that code to the application controller for your code to continue working.
=== Features
1. Store cache entries as YAML streams so the Response headers from the original
response can be returned with cache hits
2. Add a 'last-modified' header to the response to get the client to use a
get-if-modified request
3. If the client has the response we have cached, don't send it again, send a
'304 Not Modified' response to reduce data on the wire
4. Fix a bug in the original Rails code where responses other than '200 OK' are cached
(since the headers aren't cached in the original, all the clients would get
is an empty '200 OK' response from subsequent requests)
5. Allow clients to provide their own implementation of the cache key for the actions, e.g.
- application.rb
# Cache different pages for Admin and Users
def action_fragment_key(options)
url_for(options).split('://').last + "/#{admin? : 'admin' : 'user'}"
end
The options hash can be used to pass parameters in to override the current controller, and is
used by the cache expiry code to expire an action from a sweeper or a different controller than
the one the action is cached for.
6. Allow an action to specify a Time To Live for the cached item. Set 'response.time_to_live' to
the number of seconds before this cached item will be expired. If not set, the default setting
of 'never' will be used and the item will only be expired by using the regular action cache
expiry mechanism.
def my_action
@response.time_to_live = 10.minutes
...
end
7. If the ENABLE_X_SENDFILE environment variable is set, or the HTTP_ENABLE_X_SENDFILE request
header is set, and the fragment cache is set to the FileStore, then the Action Cache code
will not return the response body, but will set the X-Sendfile header in the response to
the filename of the cache entry that contains the body.
Be sure your web server is has the X-Sendfile feature enabled, otherwise you'll just get
empty responses!
Check out the lighttpd documentation for how to use the X-Sendfile feature: http://lighttpd.net/
To enable this, the ENABLE_X_SENDFILE environment variable must be set, *and* the FileStore fragment
cache must be used.
lighttpd.conf:
fastcgi.server = ( ".fcgi" =>
( "app" =>
( "min-procs" => 1,
"max-procs" => 1,
"allow-x-send-file" => "enable",
"socket" => "/tmp/app.fcgi.socket",
"bin-path" => "/path/to/app/public/dispatch.fcgi",
"bin-environment" => ( "RAILS_ENV" => "development", "ENABLE_X_SENDFILE" => "true" )
)
)
)
environment.rb:
ActionController::Base.fragment_cache_store = :file_store, "/path/to/cache/directory"
Note: The cache directory can be anywhere on your server that your web server user has read and write
access to. This should *not* be in the Rails /public directory.
8. Control whether caching occurs for an action at runtime instead of load time.
To control caching, add a method *cache_action?(action_name)* to your controller. If this
method returns true, then the action cache will work as before. If false, then caching will
not occur for this request.
e.g.
class ApplicationController < ActionController::Base
...
def cache_action?(action_name)
!admin?
end
...
end
Note: The action must still be marked for caching by adding *caches_action :action* to the controller
9. If the ENABLE_X_ACCEL_REDIRECT request header is set, and the fragment cache is set to
the FileStore, then the Action Cache code will not return the response body, but will set
the X-Accel-Redirect header in the response to the filename of the cache entry that contains the
body.
The nginx configuration must contain a 'location' section labeled 'cache', that points to the location
you have configured for your Rails fragment cache, default is RAILS_ROOT/tmp/cache. e.g:
location /cache/ {
internal;
root /path/to/rails/app/current/tmp;
}
To enable this, the ENABLE_X_SENDFILE environment variable must be set, *and* the FileStore fragment
cache must be used.
nginx.conf:
location /cache/ {
internal;
root /path/to/rails/app/current/tmp;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header "ENABLE_X_ACCEL_REDIRECT" "true";
...
}
environment.rb:
ActionController::Base.fragment_cache_store = :file_store, "/path/to/cache/directory"
Note: The cache directory can be anywhere on your server that your web server user has read and write
access to. This should *not* be in the Rails /public directory.
10. A new method 'expire_all_actions' will clear out the entire action cache contents.
11. expire_action will now work with the custom generated action cache keys, so your cache
expiry calls and sweepers will work correctly.
The expire_action call implemented here will actually use the Regexp fragment expiry call,
causing all matching cache items to be cleared. For those of you using REST, and providing
HTML, JS and XML for the same action, all three will be expired when you expire one of them
with code like:
# Expires all formats of the action
expire_action :controller => 'foo', :action => 'bar'
=== Performance
If a client requests an action whose output hasn't changed since their last request, the returning of
a 304 response instead of the full response greatly reduces the load on the server.
In my informal testing, with the X-Sendfile enabled, I was able to get about 20% more requests out of my
rails application, based on the requests-per-second displayed in the rails log. This doesn't mean the
request is faster, but that the work of delivering the content is offloaded to the web server from the
Rails app.

7
vendor/plugins/action_cache/about.yml vendored Normal file
View file

@ -0,0 +1,7 @@
author: Tom Fakes
summary: A drop-in replacement for the Rails Action Cache. Fixes some inconsistencies and adds some extra control. Use X-Sendfile or X-Accel-Redirect to send responses
homepage: http://blog.craz8.com
plugin: http://craz8.com/svn/trunk/plugins/action_cache
license: MIT
version: 0.1.4
rails_version: 1.0+

16
vendor/plugins/action_cache/index.html vendored Normal file
View file

@ -0,0 +1,16 @@
<html><head><title>Revision 53: /trunk/plugins/action_cache</title></head>
<body>
<h2>Revision 53: /trunk/plugins/action_cache</h2>
<ul>
<li><a href="../">..</a></li>
<li><a href="CHANGELOG">CHANGELOG</a></li>
<li><a href="MIT-LICENSE">MIT-LICENSE</a></li>
<li><a href="README">README</a></li>
<li><a href="about.yml">about.yml</a></li>
<li><a href="init.rb">init.rb</a></li>
<li><a href="lib/">lib/</a></li>
<li><a href="rakefile">rakefile</a></li>
<li><a href="test/">test/</a></li>
</ul>
<hr noshade><em>Powered by <a href="http://subversion.tigris.org/">Subversion</a> version 1.4.2 (r22196).</em>
</body></html>

1
vendor/plugins/action_cache/init.rb vendored Normal file
View file

@ -0,0 +1 @@
require 'action_cache'

View file

@ -0,0 +1,232 @@
require 'yaml'
require 'time'
module ActionController
class AbstractResponse #:nodoc:
attr_accessor :time_to_live
end
module Caching
module Actions
# All documentation is keeping DRY in the plugin's README
def expire_all_actions
return unless perform_caching
expire_fragment(/.*\/(META|DATA)\/.*/)
end
def expire_one_action(options)
expire_fragment(Regexp.new(".*/" + ActionCachePath.path_for(self, options) + ".*"))
end
def expire_action(options = {})
return unless perform_caching
if options[:action].is_a?(Array)
options[:action].dup.each do |action|
expire_one_action options.merge({ :action => action })
end
else
expire_one_action options
end
end
def action_fragment_key(options)
url_for(options).split('://').last
end
# Override the 1.2 ActionCachePath class, works in 1.1.x too
class ActionCachePath
attr_reader :controller, :options
class << self
def path_for(*args, &block)
new(*args).path
end
end
def initialize(controller, options = {})
@controller = controller
@options = options
end
# Override this to change behavior
def path
return @path if @path
@path = @controller.send(:action_fragment_key, @options)
add_extension!
clean!
end
def extension
@extension ||= extract_extension(controller.request.path)
end
private
def clean!
@path = @path.gsub(':', '-').gsub('?', '-')
end
def add_extension!
@path << ".#{extension}" if extension
end
def extract_extension(file_path)
# Don't want just what comes after the last '.' to accomodate multi part extensions
# such as tar.gz.
file_path[/^[^.]+\.(.+)$/, 1]
end
end
class ActionCacheFilter #:nodoc:
def self.fragment_key=(key_block)
raise "fragment_key member no longer supported - use action_fragment_key on controller instead"
end
class CacheEntry #:nodoc:
def initialize(headers, time_to_live = nil)
@headers = headers.merge({ 'cookie' => [] }) # Don't send cookies for cached responses
@expire_time = Time.now + time_to_live unless time_to_live.nil?
end
def expired?
!expire_time.nil? && Time.now > expire_time
end
attr_reader :headers, :expire_time
end
def before(controller)
if cache_entry = cached_entry(controller)
if cache_entry.expired?
remove_cache_item(controller) and return
end
if x_sendfile_enabled?(controller)
send_using_x_sendfile(cache_entry, controller)
else if x_accel_redirect_enabled?(controller)
send_using_x_accel_redirect(cache_entry, controller)
else
if client_has_latest?(cache_entry, controller)
send_not_modified(controller)
else
send_cached_response(cache_entry, controller)
end
end
end
controller.rendered_action_cache = true
return false
end
end
def after(controller)
if cache_this_request?(controller)
adjust_headers(controller)
save_to_cache(controller)
end
end
protected
def adjust_headers(controller)
if controller.response.time_to_live &&
controller.response.headers['Cache-Control'] == 'no-cache'
controller.response.headers['Cache-Control'] = "max-age=#{controller.response.time_to_live}"
end
controller.response.headers['Last-Modified'] ||= Time.now.httpdate
end
def send_cached_response(cache_entry, controller)
controller.logger.info "Send #{body_name(controller)} by response.body"
controller.response.headers = cache_entry.headers
controller.response.body = fragment_body(controller)
end
def send_not_modified(controller)
controller.logger.info "Send Not Modified"
controller.render(:text => "", :status => 304)
end
def client_has_latest?(cache_entry, controller)
requestTime = Time.rfc2822(controller.request.env["HTTP_IF_MODIFIED_SINCE"]) rescue nil
responseTime = Time.rfc2822(cache_entry.headers['Last-Modified']) rescue nil
return (requestTime and responseTime and responseTime <= requestTime)
end
def remove_cache_item(controller)
controller.expire_fragment(meta_name(controller))
controller.expire_fragment(body_name(controller))
end
def send_using_x_sendfile(cache_entry, controller)
filename = fragment_body_filename(controller)
controller.logger.info "Send #{filename} by X-Sendfile"
controller.response.headers = cache_entry.headers
controller.response.headers["X-Sendfile"] = filename
end
def send_using_x_accel_redirect(cache_entry, controller)
filename = "/cache/#{fragment_body_filename(controller)[(controller.fragment_cache_store.cache_path.length + 1)..-1]}"
controller.logger.info "Send #{filename} by X-Accel-Redirect"
controller.response.headers = cache_entry.headers
controller.response.headers["X-Accel-Redirect"] = filename
end
def fragment_body_filename(controller)
controller.fragment_cache_store.send(:real_file_path, body_name(controller))
end
def fragment_body(controller)
controller.read_fragment body_name(controller)
end
def cache_request?(controller)
controller.respond_to?(:cache_action?) ? controller.cache_action?(controller.action_name) : true
end
def cache_this_request?(controller)
@actions.include?(controller.action_name.intern) && cache_request?(controller) &&
!controller.rendered_action_cache && controller.response.headers['Status'] == '200 OK'
end
def cached_entry(controller)
if @actions.include?(controller.action_name.intern) &&
cache_request?(controller) &&
(cache = controller.read_fragment(meta_name(controller)))
return YAML.load(cache)
end
return nil
end
def save_to_cache(controller)
cache = CacheEntry.new(controller.response.headers, controller.response.time_to_live)
controller.write_fragment(body_name(controller), controller.response.body)
controller.write_fragment(meta_name(controller), YAML.dump(cache))
end
def x_sendfile_enabled?(controller)
(controller.request.env["ENABLE_X_SENDFILE"] == "true" ||
controller.request.env["HTTP_X_ENABLE_X_SENDFILE"] == "true") &&
controller.fragment_cache_store.is_a?(ActionController::Caching::Fragments::UnthreadedFileStore)
end
def x_accel_redirect_enabled?(controller)
controller.request.env["HTTP_ENABLE_X_ACCEL_REDIRECT"] == "true" &&
controller.fragment_cache_store.is_a?(ActionController::Caching::Fragments::UnthreadedFileStore)
end
def meta_name(controller)
"META/#{ActionCachePath.path_for(controller)}"
end
def body_name(controller)
"DATA/#{ActionCachePath.path_for(controller)}"
end
end
end
end
end

View file

@ -0,0 +1,9 @@
<html><head><title>Revision 53: /trunk/plugins/action_cache/lib</title></head>
<body>
<h2>Revision 53: /trunk/plugins/action_cache/lib</h2>
<ul>
<li><a href="../">..</a></li>
<li><a href="action_cache.rb">action_cache.rb</a></li>
</ul>
<hr noshade><em>Powered by <a href="http://subversion.tigris.org/">Subversion</a> version 1.4.2 (r22196).</em>
</body></html>

22
vendor/plugins/action_cache/rakefile vendored Normal file
View file

@ -0,0 +1,22 @@
require 'rake'
require 'rake/testtask'
require 'rake/rdoctask'
desc 'Default: run unit tests.'
task :default => :test
desc 'Test the Action Cache Upgrade plugin.'
Rake::TestTask.new(:test) do |t|
t.libs << 'lib'
t.pattern = 'test/**/*_test.rb'
t.verbose = true
end
desc 'Generate documentation for the Action Cache Upgrade plugin.'
Rake::RDocTask.new(:rdoc) do |rdoc|
rdoc.rdoc_dir = 'rdoc'
rdoc.title = 'Action Cache Upgrade'
rdoc.options << '--line-numbers --inline-source'
rdoc.rdoc_files.include('README')
rdoc.rdoc_files.include('lib/**/*.rb')
end

View file

@ -0,0 +1,150 @@
require "#{File.dirname(__FILE__)}/../../../../config/boot.rb"
require "#{File.dirname(__FILE__)}/../../../../config/environment.rb"
require 'action_controller/test_process'
require 'test/unit'
ActionController::Base.perform_caching = true
ActionController::Routing::Routes.reload rescue nil
require "#{File.dirname(__FILE__)}/../lib/action_cache"
class ActionCacheController < ActionController::Base
caches_action :a, :b, :c, :action_to_expire, :action_sets_cookie
attr_accessor :var
def a
response.time_to_live = 1
render :text => "Action A: Some text that will be cached: #{@var}"
end
def b
response.time_to_live = 1
render :text => "Action B: Some text that will be cached: #{@var}"
end
def c
response.time_to_live = 1
logger.info "Action C"
render :text => "Action C: Some text that will be cached: #{@var}"
end
def action_sets_cookie
cookies["one_time_only"] = "Hello!"
render :text => "Action Sets A Cookie Value"
end
def action_to_expire
logger.info "Action To Expire"
render :text => "Action To Expire: Some text that will be cached: #{@var}"
end
def clear_cache_item
expire_action :action => 'action_to_expire'
render :text => 'Cache Item Expired'
end
def clear_all_cache
expire_all_actions
render :text => 'All Cache Items Expired'
end
end
class ActionCacheTest < Test::Unit::TestCase
def setup
@controller = ActionCacheController.new
@request = ActionController::TestRequest.new
@response = ActionController::TestResponse.new
end
def test_action_cookie_not_cached
get :action_sets_cookie
assert_response :success, @response.inspect
assert_not_nil cookies["one_time_only"]
# Cache should drop the cookie and not return to the second request
get :action_sets_cookie
assert_response :success, @response.body
assert_nil cookies["one_time_only"]
end
def test_action_is_cached_without_x_sendfile
@controller.var = "nothing"
assert_not_equal "true", @request.env["ENABLE_X_SENDFILE"]
get :a
assert_response :success, @response.inspect
assert_nil @response.headers['X-Sendfile']
assert_match %r{nothing}, @response.body, "Body is not as expected: #{@response.body}"
# Make a change that the cache won't return
@controller.var = "something"
get :a
assert_response :success, @response.body
assert_nil @response.headers['X-Sendfile']
assert_match %r{nothing}, @response.body, "Body should not be changed: #{@response.body}"
end
def test_action_is_cached_with_x_sendfile
@request.env['ENABLE_X_SENDFILE'] = "true"
get :b
assert_response :success, @response.inspect
assert_nil @response.headers['X-Sendfile'], "No x-sendfile header expected: #{@response.headers.inspect}"
get :b
assert_response :success, @response.body
assert_not_nil @response.headers['X-Sendfile'], "X-sendfile header expected: #{@response.headers.inspect}"
end
def test_action_is_cached_with_accel_redirect
@request.env['HTTP_ENABLE_X_ACCEL_REDIRECT'] = "true"
get :c
assert_response :success, @response.inspect
assert_nil @response.headers['X-Accel-Redirect'], "No x-accel-redirect header expected: #{@response.headers.inspect}"
get :c
assert_response :success, @response.body
assert_not_nil @response.headers['X-Accel-Redirect'], "X-Accel-Redirect header expected: #{@response.headers.inspect}"
end
def test_expire_action
@controller.var = "nothing"
get :action_to_expire
assert_response :success, @response.inspect
assert_match %r{nothing}, @response.body, "Body is not as expected: #{@response.body}"
@controller.var = "something"
get :action_to_expire
assert_response :success, @response.body
assert_match %r{nothing}, @response.body, "Body should not be changed: #{@response.body}"
get :clear_cache_item
assert_response :success, @response.body
get :action_to_expire
assert_response :success, @response.body
assert_match %r{something}, @response.body, "Body should be changed: #{@response.body}"
end
def test_expire_all_action
@controller.var = "nothing"
get :action_to_expire
assert_response :success, @response.inspect
assert_match %r{nothing}, @response.body, "Body is not as expected: #{@response.body}"
@controller.var = "something"
get :action_to_expire
assert_response :success, @response.body
assert_match %r{nothing}, @response.body, "Body should not be changed: #{@response.body}"
get :clear_all_cache
assert_response :success, @response.body
get :action_to_expire
assert_response :success, @response.body
assert_match %r{something}, @response.body, "Body should be changed: #{@response.body}"
end
end

View file

@ -0,0 +1,9 @@
<html><head><title>Revision 53: /trunk/plugins/action_cache/test</title></head>
<body>
<h2>Revision 53: /trunk/plugins/action_cache/test</h2>
<ul>
<li><a href="../">..</a></li>
<li><a href="action_cache_test.rb">action_cache_test.rb</a></li>
</ul>
<hr noshade><em>Powered by <a href="http://subversion.tigris.org/">Subversion</a> version 1.4.2 (r22196).</em>
</body></html>