deac/couchrest_model
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

Refactoring design doc manipulation for a much simpler and more reliable approach

Browse source
This commit is contained in:
Sam Lown 2011-04-17 02:46:33 +02:00
parent 2eed3581af
commit 5805f6e27b
19 changed files with 285 additions and 195 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

40
README.md
View file

@ -378,6 +378,43 @@ Use pagination as follows:
# In your view, with the kaminari gem loaded:
paginate @posts
### Design Documents and Views
Views must be defined in a Design Document for CouchDB to be able to perform searches. Each model therefore must have its own Design Document. Deciding when to update the model's design doc is a difficult issue, as in production you don't want to be constantly checking for updates and in development maximum flexability is important. CouchRest Model solves this issue by providing the `auto_update_design_doc` configuration option and is enabled by default.
Each time a view or other design method is requested a quick GET for the design will be sent to ensure it is up to date with the latest changes. Results are cached in the current thread for the complete design document's URL, including the database, to try and limit requests. This should be fine for most projects, but dealing with multiple sub-databases may require a different strategy.
Setting the option to false will require a manual update of each model's design doc whenever you know a change has happened. This will be useful in cases when you do not want CouchRest Model to interfere with the views already store in the CouchRest database, or you'd like to deploy your own update strategy. Here's an example of a module that will update all submodules:
module CouchRestMigration
def self.update_design_docs
CouchRest::Model::Base.subclasses.each{|klass| klass.save_design_doc! if klass.respond_to?(:save_design_doc!)}
end
end
# Running this from your applications initializers would be a good idea,
# for example in Rail's application.rb or environments/production.rb:
config.after_initialize do
CouchRestMigration.update_design_docs
end
If you're dealing with multiple databases, using proxied models, or databases that are created on-the-fly, a more sophisticated approach might be required:
module CouchRestMigration
def self.update_all_design_docs
update_design_docs(COUCHREST_DATABASE)
Company.all.each do |company|
update_design_docs(company.proxy_database)
end
end
def self.update_design_docs(db)
CouchRest::Model::Base.subclasses.each{|klass| klass.save_design_doc!(db) if klass.respond_to?(:save_design_doc!(db)}
end
end
# Command to run after a capistrano migration:
$ rails runner "CouchRestMigratin.update_all_design_docs"
## Assocations
@ -546,7 +583,7 @@ such as validating for uniqueness and associations.
CouchRest Model supports a few configuration options. These can be set either for the whole Model code
base or for a specific model of your chosing. To configure globally, provide something similar to the
following in your projects loading code:
following in your projects initializers or environments:
CouchRest::Model::Base.configure do |config|
config.mass_assign_any_attribute = true
@ -563,6 +600,7 @@ Options currently avilable are:
* `mass_assign_any_attribute` - false by default, when true any attribute may be updated via the update_attributes or attributes= methods.
* `model_type_key` - 'couchrest-type' by default, is the name of property that holds the class name of each CouchRest Model.
* `auto_update_design_doc` - true by default, every time a view is requested and this option is enabled, a quick check will be performed to ensure the model's design document is up to date. When disabled, you'll need to perform the updates manually. Typically, this option should be enabled in development, and disabled in production. See the View section for more details.
## Notable Issues

2
history.txt
View file

@ -2,6 +2,8 @@
* Minor enhancements:
* Adding "couchrest-hash" to Design Docs with aim to improve view update handling.
* Major changes to the way design document updates are handled internally.
* Added "auto_update_design_doc" configuration option.
* Using #descending on View object will automatically swap startkey with endkey.
== 1.1.0.beta2

11
lib/couchrest/model/collection.rb
View file

@ -1,7 +1,13 @@
module CouchRest
module Model
# Warning! The Collection module is seriously depricated.
# Use the new Design Views instead, as this code copies many other parts
# of CouchRest Model.
#
# Expect this to be removed soon.
#
module Collection
def self.included(base)
base.extend(ClassMethods)
end
@ -131,6 +137,9 @@ module CouchRest
else
@view_name = "#{design_doc}/#{view_name}"
end
# Save the design doc, ready for use
@container_class.save_design_doc(@database)
end
# See Collection.paginate

4
lib/couchrest/model/configuration.rb
View file

@ -10,10 +10,12 @@ module CouchRest
included do
add_config :model_type_key
add_config :mass_assign_any_attribute
add_config :auto_update_design_doc
configure do |config|
config.model_type_key = 'couchrest-type' # 'model'?
config.mass_assign_any_attribute = false
config.auto_update_design_doc = true
end
end

147
lib/couchrest/model/design_doc.rb
View file

@ -3,19 +3,13 @@ module CouchRest
module Model
module DesignDoc
extend ActiveSupport::Concern
module ClassMethods
def design_doc
@design_doc ||= ::CouchRest::Design.new(default_design_doc)
end
# Use when something has been changed, like a view, so that on the next request
# the design docs will be updated (if changed!)
def req_design_doc_refresh
@design_doc_fresh = { }
end
def design_doc_id
"_design/#{design_doc_slug}"
end
@ -24,6 +18,80 @@ module CouchRest
self.to_s
end
def design_doc_full_url(db = database)
"#{db.uri}/#{design_doc_id}"
end
# Retreive the latest version of the design document directly
# from the database. This is never cached and will return nil if
# the design is not present.
#
# Use this method if you'd like to compare revisions [_rev] which
# is not stored in the normal design doc.
def stored_design_doc(db = database)
db.get(design_doc_id)
rescue RestClient::ResourceNotFound
nil
end
# Save the design doc onto a target database in a thread-safe way,
# not modifying the model's design_doc
#
# See also save_design_doc! to always save the design doc even if there
# are no changes.
def save_design_doc(db = database, force = false)
update_design_doc(db, force)
end
# Force the update of the model's design_doc even if it hasn't changed.
def save_design_doc!(db = database)
save_design_doc(db, true)
end
private
def design_doc_cache
Thread.current[:couchrest_design_cache] ||= {}
end
def design_doc_cache_checksum(db)
design_doc_cache[design_doc_full_url(db)]
end
def set_design_doc_cache_checksum(db, checksum)
design_doc_cache[design_doc_full_url(db)] = checksum
end
# Writes out a design_doc to a given database if forced
# or the stored checksum is not the same as the current
# generated checksum.
#
# Returns the original design_doc provided, but does
# not update it with the revision.
def update_design_doc(db, force = false)
return design_doc unless force || auto_update_design_doc
# Grab the design doc's checksum
checksum = design_doc.checksum!
# If auto updates enabled, check checksum cache
return design_doc if auto_update_design_doc && design_doc_cache_checksum(db) == checksum
# Load up the stored doc (if present), update, and save
saved = stored_design_doc(db)
if saved
if force || saved['couchrest-hash'] != checksum
saved.merge!(design_doc)
db.save_doc(saved)
end
else
db.save_doc(design_doc)
design_doc.delete('_rev') # Prevent conflicts, never store rev as DB specific
end
# Ensure checksum cached for next attempt if using auto updates
set_design_doc_cache_checksum(db, checksum) if auto_update_design_doc
design_doc
end
def default_design_doc
{
"_id" => design_doc_id,
@ -40,68 +108,7 @@ module CouchRest
}
end
# DEPRECATED
# use stored_design_doc to retrieve the current design doc
def all_design_doc_versions(db = database)
db.documents :startkey => "_design/#{self.to_s}",
:endkey => "_design/#{self.to_s}-\u9999"
end
# Retreive the latest version of the design document directly
# from the database.
def stored_design_doc(db = database)
db.get(design_doc_id) rescue nil
end
alias :model_design_doc :stored_design_doc
def refresh_design_doc(db = database)
raise "Database missing for design document refresh" if db.nil?
unless design_doc_fresh(db)
save_design_doc(db)
design_doc_fresh(db, true)
end
end
# Save the design doc onto a target database in a thread-safe way,
# not modifying the model's design_doc
#
# See also save_design_doc! to always save the design doc even if there
# are no changes.
def save_design_doc(db = database, force = false)
update_design_doc(Design.new(design_doc), db, force)
end
# Force the update of the model's design_doc even if it hasn't changed.
def save_design_doc!(db = database)
save_design_doc(db, true)
end
protected
def design_doc_fresh(db, fresh = nil)
@design_doc_fresh ||= {}
if fresh.nil?
@design_doc_fresh[db.uri] || false
else
@design_doc_fresh[db.uri] = fresh
end
end
# Writes out a design_doc to a given database, returning the
# updated design doc
def update_design_doc(design_doc, db, force = false)
design_doc['couchrest-hash'] = design_doc.checksum
saved = stored_design_doc(db)
if saved
if force || saved['couchrest-hash'] != design_doc['couchrest-hash']
saved.merge!(design_doc)
db.save_doc(saved)
end
else
db.save_doc(design_doc)
end
design_doc
end
end # module ClassMethods

2
lib/couchrest/model/designs.rb
View file

@ -28,8 +28,6 @@ module CouchRest
mapper.create_view_method(:all)
mapper.instance_eval(&block) if block_given?
req_design_doc_refresh
end
# Override the default page pagination value:

32
lib/couchrest/model/proxyable.rb
View file

@ -48,7 +48,6 @@ module CouchRest
end
# Base
def new(*args)
proxy_update(model.new(*args))
end
@ -56,7 +55,7 @@ module CouchRest
def build_from_database(doc = {})
proxy_update(model.build_from_database(doc))
end
def method_missing(m, *args, &block)
if has_view?(m)
if model.respond_to?(m)
@ -73,32 +72,32 @@ module CouchRest
end
super
end
# DocumentQueries
def all(opts = {}, &block)
proxy_update_all(@model.all({:database => @database}.merge(opts), &block))
end
def count(opts = {})
@model.count({:database => @database}.merge(opts))
end
def first(opts = {})
proxy_update(@model.first({:database => @database}.merge(opts)))
end
def last(opts = {})
proxy_update(@model.last({:database => @database}.merge(opts)))
end
def get(id)
proxy_update(@model.get(id, @database))
end
alias :find :get
# Views
def has_view?(view)
@model.has_view?(view)
end
@ -106,27 +105,22 @@ module CouchRest
def view_by(*args)
@model.view_by(*args)
end
def view(name, query={}, &block)
proxy_update_all(@model.view(name, {:database => @database}.merge(query), &block))
end
def first_from_view(name, *args)
# add to first hash available, or add to end
(args.last.is_a?(Hash) ? args.last : (args << {}).last)[:database] = @database
proxy_update(@model.first_from_view(name, *args))
end
# DesignDoc
def design_doc
@model.design_doc
end
def refresh_design_doc(db = nil)
@model.refresh_design_doc(db || @database)
end
def save_design_doc(db = nil)
@model.save_design_doc(db || @database)
end

15
lib/couchrest/model/support/couchrest_database.rb
View file

@ -1,15 +0,0 @@
CouchRest::Database.class_eval do
alias :delete_orig! :delete!
def delete!
clear_model_fresh_cache
delete_orig!
end
# If the database is deleted, ensure that the design docs will be refreshed.
def clear_model_fresh_cache
::CouchRest::Model::Base.subclasses.each{|klass| klass.req_design_doc_refresh if klass.respond_to?(:req_design_doc_refresh)}
end
end

9
lib/couchrest/model/support/couchrest_design.rb
View file

@ -1,18 +1,19 @@
CouchRest::Design.class_eval do
# Calculate a checksum of the Design document. Used for ensuring the latest
# version has been sent to the database.
# Calculate and update the checksum of the Design document.
# Used for ensuring the latest version has been sent to the database.
#
# This will generate an flatterned, ordered array of all the elements of the
# design document, convert to string then generate an MD5 Hash. This should
# result in a consisitent Hash accross all platforms.
#
def checksum
def checksum!
# create a copy of basic elements
base = self.dup
base.delete('_id')
base.delete('_rev')
base.delete('couchrest-hash')
result = nil
flatten =
lambda {|r|
@ -26,7 +27,7 @@ CouchRest::Design.class_eval do
end
}).call(r)
}
Digest::MD5.hexdigest(flatten.call(base).sort.join(''))
self['couchrest-hash'] = Digest::MD5.hexdigest(flatten.call(base).sort.join(''))
end
end

20
lib/couchrest/model/views.rb
View file

@ -81,7 +81,6 @@ module CouchRest
end
keys.push opts
design_doc.view_by(*keys)
req_design_doc_refresh
end
# returns stored defaults if there is a view named this in the design doc
@ -99,9 +98,9 @@ module CouchRest
def view(name, query={}, &block)
query = query.dup # Modifications made on copy!
db = query.delete(:database) || database
refresh_design_doc(db)
query[:raw] = true if query[:reduce]
raw = query.delete(:raw)
save_design_doc(db)
fetch_view_with_docs(db, name, query, raw, &block)
end
@ -140,23 +139,10 @@ module CouchRest
def fetch_view(db, view_name, opts, &block)
raise "A view needs a database to operate on (specify :database option, or use_database in the #{self.class} class)" unless db
retryable = true
begin
design_doc.view_on(db, view_name, opts, &block)
# the design doc may not have been saved yet on this database
rescue RestClient::ResourceNotFound => e
if retryable
save_design_doc(db)
retryable = false
retry
else
raise e
end
end
design_doc.view_on(db, view_name, opts, &block)
end
end # module ClassMethods
end
end
end

1
lib/couchrest_model.rb
View file

@ -45,7 +45,6 @@ require "couchrest/model/designs"
require "couchrest/model/designs/view"
# Monkey patches applied to couchrest
require "couchrest/model/support/couchrest_database"
require "couchrest/model/support/couchrest_design"
# Core Extensions
require "couchrest/model/core_extensions/hash"

1
spec/couchrest/base_spec.rb
View file

@ -262,7 +262,6 @@ describe "Model Base" do
describe "counting all instances of a model" do
before(:each) do
@db = reset_test_db!
# WithTemplateAndUniqueID.req_design_doc_refresh
end
it ".count should return 0 if there are no docuemtns" do

1
spec/couchrest/collection_spec.rb
View file

@ -5,7 +5,6 @@ describe "Collections" do
before(:all) do
reset_test_db!
Article.refresh_design_doc
titles = ["very uniq one", "really interesting", "some fun",
"really awesome", "crazy bob", "this rocks", "super rad"]
titles.each_with_index do |title,i|

132
spec/couchrest/design_doc_spec.rb
View file

@ -6,23 +6,93 @@ require File.join(FIXTURE_PATH, 'more', 'article')
describe "Design Documents" do
before :all do
reset_test_db!
end
describe "CouchRest Extension" do
it "should have created a checksum method" do
::CouchRest::Design.new.should respond_to(:checksum)
it "should have created a checksum! method" do
::CouchRest::Design.new.should respond_to(:checksum!)
end
it "should calculate a consistent checksum for model" do
WithTemplateAndUniqueID.design_doc.checksum.should eql('7786018bacb492e34a38436421a728d0')
WithTemplateAndUniqueID.design_doc.checksum!.should eql('7786018bacb492e34a38436421a728d0')
end
it "should calculate checksum for complex model" do
Article.design_doc.checksum.should eql('1e6c315853cd5ff10e5c914863aee569')
Article.design_doc.checksum!.should eql('1e6c315853cd5ff10e5c914863aee569')
end
it "should cache the generated checksum value" do
Article.design_doc.checksum!
Article.design_doc['couchrest-hash'].should_not be_blank
end
end
describe "class methods" do
describe ".design_doc" do
it "should provide Design document" do
Article.design_doc.should be_a(::CouchRest::Design)
end
end
describe ".design_doc_id" do
it "should provide a reasonable id" do
Article.design_doc_id.should eql("_design/Article")
end
end
describe ".design_doc_slug" do
it "should provide slug part of design doc" do
Article.design_doc_slug.should eql('Article')
end
end
describe ".design_doc_full_url" do
it "should provide complete url" do
Article.design_doc_full_url.should eql("#{DB.uri}/_design/Article")
end
it "should provide complete url for new DB" do
db = mock("Database")
db.should_receive(:uri).and_return('db')
Article.design_doc_full_url(db).should eql("db/_design/Article")
end
end
describe ".stored_design_doc" do
it "should load a stored design from the database" do
Article.by_date
Article.stored_design_doc['_rev'].should_not be_blank
end
it "should return nil if not already stored" do
WithDefaultValues.stored_design_doc.should be_nil
end
end
describe ".save_design_doc" do
it "should call up the design updater" do
Article.should_receive(:update_design_doc).with('db', false)
Article.save_design_doc('db')
end
end
describe ".save_design_doc!" do
it "should call save_design_doc with force" do
Article.should_receive(:save_design_doc).with('db', true)
Article.save_design_doc!('db')
end
end
end
describe "basics" do
before :all do
reset_test_db!
end
it "should have been instantiated with views" do
d = Article.design_doc
d['views']['all']['map'].should include('Article')
@ -54,6 +124,13 @@ describe "Design Documents" do
written_at += 24 * 3600
end
end
it "will send request for the saved design doc on view request" do
reset_test_db!
Article.should_receive(:stored_design_doc).and_return(nil)
Article.by_date
end
it "should have generated a design doc" do
Article.design_doc["views"]["by_date"].should_not be_nil
end
@ -68,19 +145,54 @@ describe "Design Documents" do
design = Article.design_doc
view = design['views']['by_date']['map']
design['views']['by_date']['map'] = view + ' ' # little bit of white space
Article.req_design_doc_refresh
Article.by_date
orig = Article.stored_design_doc
orig['views']['by_date']['map'].should eql(Article.design_doc['views']['by_date']['map'])
Article.stored_design_doc['_rev'].should_not eql(orig['_rev'])
orig['views']['by_date']['map'].should_not eql(Article.design_doc['views']['by_date']['map'])
end
it "should not save design doc if not changed" do
Article.by_date
orig = Article.stored_design_doc['_rev']
Article.req_design_doc_refresh
Article.by_date
Article.stored_design_doc['_rev'].should eql(orig)
end
end
describe "when auto_update_design_doc false" do
before :all do
Article.auto_update_design_doc = false
Article.save_design_doc!
end
after :all do
Article.auto_update_design_doc = true
end
it "will not send a request for the saved design doc" do
Article.should_not_receive(:stored_design_doc)
Article.by_date
end
it "will not update stored design doc if view changed" do
Article.by_date
orig = Article.stored_design_doc
design = Article.design_doc
view = design['views']['by_date']['map']
design['views']['by_date']['map'] = view + ' '
Article.by_date
Article.stored_design_doc['_rev'].should eql(orig['_rev'])
end
it "will update stored design if forced" do
Article.by_date
orig = Article.stored_design_doc
design = Article.design_doc
view = design['views']['by_date']['map']
design['views']['by_date']['map'] = view + ' '
Article.save_design_doc!
Article.stored_design_doc['_rev'].should_not eql(orig['_rev'])
end
end
end
describe "lazily refreshing the design document" do
@ -90,11 +202,7 @@ describe "Design Documents" do
end
it "should not save the design doc twice" do
WithTemplateAndUniqueID.all
WithTemplateAndUniqueID.req_design_doc_refresh
WithTemplateAndUniqueID.refresh_design_doc
rev = WithTemplateAndUniqueID.design_doc['_rev']
WithTemplateAndUniqueID.req_design_doc_refresh
WithTemplateAndUniqueID.refresh_design_doc
WithTemplateAndUniqueID.design_doc['_rev'].should eql(rev)
end
end

5
spec/couchrest/designs_spec.rb
View file

@ -31,11 +31,6 @@ describe "Design" do
DesignModel.design { foo }
end
it "should request a design refresh" do
DesignModel.should_receive(:req_design_doc_refresh)
DesignModel.design() { }
end
it "should work even if a block is not provided" do
lambda { DesignModel.design }.should_not raise_error
end

11
spec/couchrest/proxyable_spec.rb
View file

@ -233,17 +233,6 @@ describe "Proxyable" do
@obj.design_doc
end
describe "#refresh_design_doc" do
it "should be proxied without database arg" do
Cat.should_receive(:refresh_design_doc).with('database')
@obj.refresh_design_doc
end
it "should be proxied with database arg" do
Cat.should_receive(:refresh_design_doc).with('db')
@obj.refresh_design_doc('db')
end
end
describe "#save_design_doc" do
it "should be proxied without args" do
Cat.should_receive(:save_design_doc).with('database')

22
spec/couchrest/subclass_spec.rb
View file

@ -61,37 +61,27 @@ describe "Subclassing a Model" do
validated_fields.should_not include(:extension_code)
validated_fields.should_not include(:job_title)
end
it "should inherit default property values" do
@card.bg_color.should == '#ccc'
end
it "should be able to overwrite a default property" do
DesignBusinessCard.new.bg_color.should == '#eee'
end
it "should have a design doc slug based on the subclass name" do
Course.refresh_design_doc
OnlineCourse.design_doc_slug.should =~ /^OnlineCourse/
end
it "should have its own design_doc_fresh" do
Animal.refresh_design_doc
Dog.send(:design_doc_fresh, Dog.database).should_not == true
Dog.refresh_design_doc
Dog.send(:design_doc_fresh, Dog.database).should == true
end
it "should not add views to the parent's design_doc" do
Course.design_doc['views'].keys.should_not include('by_url')
end
it "should not add the parent's views to its design doc" do
Course.refresh_design_doc
OnlineCourse.refresh_design_doc
OnlineCourse.design_doc['views'].keys.should_not include('by_title')
end
it "should have an all view with a guard clause for model == subclass name in the map function" do
OnlineCourse.design_doc['views']['all']['map'].should =~ /if \(doc\['#{OnlineCourse.model_type_key}'\] == 'OnlineCourse'\)/
end

23
spec/couchrest/view_spec.rb
View file

@ -19,11 +19,11 @@ describe "Model views" do
# NOTE! Add more unit tests!
describe "#view" do
it "should not alter original query" do
options = { :database => DB }
view = Article.view('by_date', options)
options[:database].should_not be_nil
options[:database].should eql(DB)
end
end
@ -266,19 +266,7 @@ describe "Model views" do
u = Unattached.last :database=>@db
u.title.should == "aaa"
end
it "should barf on all_design_doc_versions if no database given" do
lambda{Unattached.all_design_doc_versions}.should raise_error
end
it "should be able to cleanup the db/bump the revision number" do
# if the previous specs were not run, the model_design_doc will be blank
Unattached.use_database DB
Unattached.view_by :questions
Unattached.by_questions(:database => @db)
original_revision = Unattached.model_design_doc(@db)['_rev']
Unattached.save_design_doc!(@db)
Unattached.model_design_doc(@db)['_rev'].should_not == original_revision
end
end
describe "a model with a compound key view" do
@ -346,7 +334,7 @@ describe "Model views" do
before(:each) do
reset_test_db!
Article.by_date
@original_doc_rev = Article.model_design_doc['_rev']
@original_doc_rev = Article.stored_design_doc['_rev']
@design_docs = Article.database.documents :startkey => "_design/", :endkey => "_design/\u9999"
end
it "should not create a design doc on view definition" do
@ -355,10 +343,9 @@ describe "Model views" do
newdocs["rows"].length.should == @design_docs["rows"].length
end
it "should create a new version of the design document on view access" do
ddocs = Article.all_design_doc_versions["rows"].length
Article.view_by :updated_at
Article.by_updated_at
@original_doc_rev.should_not == Article.model_design_doc['_rev']
@original_doc_rev.should_not == Article.stored_design_doc['_rev']
Article.design_doc["views"].keys.should include("by_updated_at")
end
end

2
spec/spec_helper.rb
View file

@ -22,6 +22,8 @@ end
def reset_test_db!
DB.recreate! rescue nil
# Reset the Design Cache
Thread.current[:couchrest_design_cache] = {}
DB
end