Merge branch 'bzr/golem' of /Users/distler/Sites/code/instiki

Credit to Andrew Stacey

CHANGELOG

@@ -1,3 +1,195 @@
+N.B.: You *must* run
+
+     ruby bundle
+     ruby bundle exec rake upgrade_instiki
+
+after installing the new software, to enjoy the benefits of this new version.
+
+------------------------------------------------------------------------------
+* 0.19.3
+
+New Features:
+* Source view for Revisions
+* Rails updated to 2.3.14 (Security)
+* itextomml updated to 1.4.6
+* Replace REXML with Nokogiri in Maruku and
+  in xhtml_safe_sanitize().
+  (Huge speedup in rendering long pages)
+* MathJax updated to 1.1a final
+
+Bugs Fixed:
+* Bundler upgraded to 1.0.18
+* Fix null search bug
+* Better text/html serialization (thank you, Nokogiri)
+* Fix Maruku footnote backlink (reported by Shamaoke)
+* Fix Maruku link bug
+* Fix Maruku image title bug
+* Fix Maruku hrule, email address and header bugs
+* Fix Maruku bold-in-italics bug
+* Fix Maruku empty list-item bug
+
+------------------------------------------------------------------------------
+* 0.19.2
+
+New Features:
+* MathJax rendering for non-MathML capable browsers.
+* RedCloth (Textile) upgraded to 4.x (now handled
+  by Bundler).
+* Bundler upgraded to 1.0.7
+* Rails updated to 2.3.11
+
+Bugs Fixed:
+* Redirects and categories of included pages should
+  not be inherited. (Suggestion of Andrew Stacey).
+* Bug in Maruku equation handling (reported by Andrew Stacey).
+* SVG-Edit updates and bug-fixes.
+* Bug in editing S5 slideshows.
+* Unvendor Rack
+* Fix Maruku list-parsing bug (reported by Shamaoke)
+* Validate Web address (Reported by Richard Marquez).
+* Fix a well-formedness bug
+
+------------------------------------------------------------------------------
+* 0.19.1
+
+New Features:
+* From the "All" or Category listings, you can export selected pages (in any
+  desired order) to a single LaTeX file.
+* LaTeX export supports \array{} (with no options) and a LaTeX-style optional
+  argument for \sqrt[]{}. The latter requires itextomml 1.4.5 or later.
+* Updated to itextomml 1.4.5 (a bunch of new itex language features).
+* Updated to Rails 2.3.10. (There were security issues in 2.3.9 which, happily,
+  did NOT affect Instiki 0.19. But 2.3.10 has other improvements, as well.)
+
+Bugs Fixed:
+* Several SVG-Edit bugs fixed.
+* Removed some superfluous junk, to slim down the distribution (cuts the size of
+  the compressed .tar.gz nearly in half).
+
+------------------------------------------------------------------------------
+* 0.19
+
+New Features:
+
+* WYSIWYG SVG editing (via SVG-edit)
+* One-click S5 templates
+* Itex2MML is now a Rubygem. Latest is itextomml-1.4.2.
+* Rails Metal itex endpoint
+* HTML5 support
+* Support IALs on Markdown list items
+* Updated to Rails 2.3.9 and Erubis (now at 2.6.6)
+* Updated for Rack 1.2.1, sqlite3-ruby 1.3.1
+* Manages dependencies using Bundler. Before running Instiki for the first time
+  (and whenever you update), run
+  
+     ruby bundle
+     rake upgrade_instiki
+
+  from the instiki directory. (You may need to run
+  
+     ruby bundle exec rake upgrade_instiki
+
+  instead, if you get a complaint about your version of rake.)
+
+Bugs Fixed:
+
+* Works with Ruby 1.9.2
+* Fixed a bug in non-Latin WikiWord processing. (Reported by Alexander Hambug)
+* Fixed Cyrillic WikiWord support.
+* More informative dnsbl lookup responses (suggested by Toby Bartels)
+* Fixed a bug in LaTeX output
+* No longer conflicts with sqlite3-ruby 1.3.x Rubygem
+* Fixed some Category listing bugs
+* Fixed an escaping bug in 'new' and 'edit' templates. (Reported by Toby Bartels)
+* Allow special characters ('.', '/', etc) in page names.
+* Fix BlahTeX/PNG path, so equations render in diff and
+  previous revision pages.
+* Fix HTML Export feature so that uploaded files are
+  included, stylesheets load, etc.
+* Uploaded files inclided in Markup Export.
+* Fix Print View, so that uploaded images work.
+* Fix some more Ruby 1.9 isues.
+* Prevent page from being renamed to null.
+* Fix Migration to work under PostgreSQL (from J. Zellman).
+* Updated vendored plugins
+
+------------------------------------------------------------------------------
+* 0.18.1
+
+New Features:
+
+* (Markdown-Extra syle) fenced codeblocks. [From Jason Blevins]
+* Fortran syntax colouring. [From Jason Blevins]
+
+Bugs Fixed:
+
+* Fixed some Ruby 1.9 encoding issues, with unicode page names,
+  author names and categories.
+* Better display of inter-web wikilinks
+* various syntax-colouring fixes
+* Corrected length of wiki_references referenced_name (affects
+  MySQL users)
+
+------------------------------------------------------------------------------
+* 0.18
+
+New Features:
+
+* Syntax colouring: 'ansic', 'javascript', 'sqlite', 'yaml' and 'css' modes,
+  in addition to the existing 'html', 'xml', and 'ruby' modes,
+* Source view [suggested by Andrew Stacey]
+* Auto-resizing Textareas scale to fit viewing area.
+* Instiki upgraded to Rails 2.3.5 and Rack 1.1.
+* Now runs on Ruby 1.9. (If you're a Passenger user, you may need to upgrade to Passenger
+  2.2.8, which works around some bugs in Ruby 1.9.1.) 
+* Upgraded for itex2MML 1.3.19 (which works under Ruby 1.9, and has several new feautures,
+  relative to 1.3.15).
+
+Bugs Fixed:
+
+* Fixed a CSS bug, which screwed up printing (unless you used the "Print" view).
+* Fixed a well-formedness bug in the page-name truncation algorithm [reported by Toby Bartels]
+* Fixed a cache-sweeping bug [reported by Toby Bartels]
+* Better accessibility.
+* Improved log rotation under Passenger.
+* Omit a (seemingly superfluous) javascript hack which causes Gecko-based browsers to request
+     /my_wiki/s5/null
+  when they load an s5 slideshow.
+* Upgraded vendored sqlite3-ruby and rubyzip
+* Move files when renaming a web (so that links to uploaded files don't break).
+* Many Ruby 1.9 fixes, including removing the html5lib Sanitizer.
+
+------------------------------------------------------------------------------
+* 0.17.3
+
+The most important facet of this release is a small change in the database
+schema. Previously, people migrating from the default SQLite3 database to MySQL
+ran the risk of silent data loss, because MySQL had a more strict interpretation
+of the column types in the database. The new schema will prevent such problems.
+
+     rake upgrade_instiki
+
+will seamlessly upgrade your existing database to the new schema.
+
+New Features:
+* Passenger support (including X-Sendfile support, if the Apache mod_xsendfile
+  module is installed).
+* Update for itex2MML 1.3.13. (You should upgrade your itex2MML to the latest
+  version, too.)
+
+Bugs Fixed:
+* Refactored the Web model (from James Herdman).
+* Clean malformed utf-8 strings, rather than complaining about them.
+* Updated location of Textile help, since _why_the_lucky_stiff left the 'net.
+* Fixed a TeX rendering bug.
+* Updated list of XHTML+MathML named entities to match W3C Working Draft.
+* Refactored the Sanitizer (speedup).
+* Fix S5 Slideshows for non-root Instiki URLs.
+* Work around a Rails flash bug.
+* Links from published webs should work right (finally?).
+* An important database migration for MySQL users.
+
+------------------------------------------------------------------------------
 * 0.17.2
 
 Security: Updated to Rails 2.3.4

Gemfile

@@ -0,0 +1,12 @@
+source "http://rubygems.org"
+gem "sqlite3-ruby", :require => "sqlite3"
+gem "itextomml", ">=1.4.7"
+gem "rack", ">=1.1.0"
+gem "mongrel", ">=1.2.0.pre2"
+gem "rubyzip"
+gem "RedCloth", ">=4.0.0"
+gem "erubis"
+gem "nokogiri"
+gem "rake"
+gem "rdoc"
+gem "json"

README

@@ -1,7 +1,7 @@
 
 = Instiki
 
-Instiki is a wiki clone so pretty and easy to set up, you'll wonder if it’s really a wiki. Runs on Rails and focuses on portability and stability. Supports file uploads, PDF export, RSS, multiple users and password protection. Some use Instiki as a CMS (Content Management System) because of it's ability to export static pages.
+Instiki is a wiki clone so pretty and easy to set up, you'll wonder if it’s really a wiki. Runs on Rails and focuses on portability and stability. Supports file uploads, PDF export, RSS, multiple users and password protection. Some use Instiki as a CMS (Content Management System) because of its ability to export static pages.
 
 Instiki lowers the barriers of interest for when you might consider using a wiki. It's so simple to get running that you'll find yourself using it for anything -- taking notes, brainstorming, organizing a gathering.
 
@@ -15,33 +15,32 @@ Instiki on BeOS, Amiga OS, OS2, Zeta OS and support for various exotic Platforms
 
 == 3 easy Steps to get the Instiki experience
 
-  Step 1. Get Ruby, Download Instiki
+  Step 1. Get Instiki and run "ruby bundle"
   Step 2. Run "instiki"
   Step 3. Chuckle... "There's no step three!" (TM)
  
 
 == Details
 
-You need at least Ruby Version 1.8.4 installed on your System. The second dependency is a Database System, but don't worry, maybe you are already served.
+You need at least Ruby Version 1.8.6, and Rubygems 1.3.6, installed on your System. The second dependency is a Database System, but don't worry, the default sqlite3 will be installed for you, if it's not already installed. You can also use any other database system (MySQL, PostgreSQL, ...) supported by Rails.
 
 
 === If you are on Windows
 
 - Get the *Ruby One-Click Installer - Windows* http://rubyforge.org/projects/rubyinstaller
+- Get Development Kit http://github.com/oneclick/rubyinstaller/wiki/development-kit
+- In the Instiki directory, execute "ruby bundle"
 - double-click instiki.bat or instiki.cmd and there you go!
 
-if you are running Windows 95, 98 or ME and cannot get instiki to run, try Version 0.11.pl1 which is the last instiki Version to support that old-style OS's. Please update to some Unix-OS or complain to the Ruby on Rails List at http://www.ruby-forum.com/forum/3 (Rails does not support your old Windows.)
-
-
 === If you are on Mac OSX
 
-On Leopard, you are all set.
+On Leopard and Snow Leopard, you are all set.
+- run "sudo gem update --system" via the command-line.
+- run "ruby bundle" in the instiki directory.
+- run "ruby instiki" and there you go!
 
 Tiger ships with a really old Ruby Version (1.8.2) and a broken Readline Library you have to
 - use the Ruby One-Click-Installer for OSX ( http://rubyosx.com ) if you don't already have macports' Ruby
-- make sure you read http://instiki.5uper.net/instiki/show/SQLite+issues+on+OSX
-- run "ruby instiki.rb" via command-line in the directory 
-
 
 === If you are on Linux
 
@@ -49,7 +48,12 @@ Tiger ships with a really old Ruby Version (1.8.2) and a broken Readline Library
 === Any other System
 
 - get Ruby for your System, compile if nessesary: http://ruby-lang.org
+- Depending on the version of Rubygems that came with your Ruby, you may need to
+
+    sudo gem update --system
+    
 - get SQLite or compile from http://sqlite.org (you can also use mysql or any other supported database system if you want)
+- run "ruby bundle"
 - run instiki
 
 You're now running a perfectly suitable wiki on port 2500 that'll present you with one-step setup, followed by a textarea for the home page on http://localhost:2500
@@ -68,14 +72,16 @@ You're now running a perfectly suitable wiki on port 2500 that'll present you wi
 * Five markup choices:
    Markdown-based choices [http://daringfireball.net/projects/markdown/syntax]:
      Markdown+itex2MML (the default; requires itex2MML) 
-     Markdown+BlahTeX/PNG (requires blahtex and a working TeX installation
+     Markdown+BlahTeX/PNG (requires blahtex and a working TeX installation)
      Markdown
    Textile [http://www.textism.com/tools/textile]
    RDoc [http://rdoc.sourceforge.net/doc]
+* Support for Math (using itex syntax [http://golem.ph.utexas.edu/~distler/blog/itex2MMLcommands.html]) 
+* Support for WYSIWYG SVG editing -- embed SVG graphics right in your wiki page.
 * Embedded webserver: uses Mongrel (if installed), or the bundled WEBrick webserver (if not).
 * Internationalization: Wiki words in any latin, greek, cyrillian, or armenian characters
 * Color diffs: Track changes through revisions
-* Runs on SQLite per default, can be configured to run on PostgreSQL, MySQL, DB2, Firebird, Openbase, Oracle, SQL Server or Sybase
+* Runs on SQLite3 per default, can be configured to run on PostgreSQL, MySQL, DB2, Firebird, Openbase, Oracle, SQL Server or Sybase
 
 
 == Command-line options:
@@ -87,56 +93,19 @@ You're now running a perfectly suitable wiki on port 2500 that'll present you wi
 
  * See CHANGELOG
 
-== Migrating from Instiki 0.11-0.12 to 0.16.3
-
-   rake upgrade_instiki
-
-== Migrating Instiki 0.10.2 storage to Instiki 0.11.0 database
-
-1. Install Instiki 0.11 and check that it works (you should be able to create a web, edit and save a HomePage)
-2. Execute 
-      ruby script\import_storage \
-          -t /full/path/to/instiki0.10/storage \
-          -i /full/path/to/instiki0.10/installation \
-          -d sqlite (or mysql, or postgres, depending on what you use) \
-          -o instiki_import.sql
-   for example (Windows):
-   ruby script\import_storage -t c:\instiki-0.10.2\storage\2500 -i c:\instiki-0.10.2 -d sqlite -o instiki_import.sql
-3. This will produce instiki_import.sql file in the current working directory. 
-   Open it in a text editor and inspect carefully.
-4. Connect to your production database (e.g., 'sqlite3 db\prod.db'), 
-   and have it execute instiki_import.sql (e.g., '.read instiki_import.sql')
-5. Execute ruby script\reset_references 
-   (this script parses all pages for crosslinks between them, so it may take a few minutes)
-6. Restart Instiki
-7. Go over some pages, especially those with a lot of complex markup, and see if anything is broken.
-
-The most common migration problem is this: if you open All Pages and see a lot of orphaned pages, 
-you forgot to run ruby script\reset_references after importing the data.
-
-
-===Upgrading from Instiki-AR Beta 1
-
-In Beta 2, we switch to ActiveRecord:Migrations. Therefore:
-1. Back up your production database.
-2. Open command-line session to your database and execute: 
-  create table schema_info (version integer(11)); 
-  insert into schema_info (version) values (1);
-3. Go back to the shell, change directory to the new Instiki and execute "rake migrate".
-
-Step 2 creates a table that tells to ActiveRecord:Migrations that the current version 
-of this database is 1 (corresponding to Beta 1), and step 3 makes it up-to-date with
-the current version of Instiki.
-
+== Migrating from Instiki 0.11-0.18 to 0.19
 
+   ruby bundle
+   ruby bundle exec rake upgrade_instiki
+   
 == Download the latest release from:
 
 * http://rubyforge.org/project/showfiles.php?group_id=186
 
 
-== Visit the "official" Instiki wiki:
+== Visit the Instiki wiki:
 
-* http://instiki.org
+* http://golem.ph.utexas.edu/wiki/instiki/
 
 
 == License:

UPGRADING

@@ -0,0 +1,21 @@
+UPGRADING
+=========
+
+See the upgrading instructions 
+
+     http://golem.ph.utexas.edu/wiki/instiki/show/Upgrading
+
+for detailed instructions.
+
+At a minimum, you need to backup your database.
+After installing the new software and restoring your database, you need to run
+
+     ruby bundle
+     ruby bundle exec rake upgrade_instiki
+
+from the commandline, to complete the upgrade. Doing a
+
+
+     ruby bundle update
+
+will update the installed gems (used by Instiki) to the latest versions.

app/controllers/admin_controller.rb

@@ -138,15 +138,4 @@ class AdminController < ApplicationController
     redirect_to :back
   end
 
-private
-
-  def is_post
-    unless (request.post? || ENV["RAILS_ENV"] == "test")
-      headers['Allow'] = 'POST'
-      render(:status => 405, :text => 'You must use an HTTP POST', :layout => 'error')
-      return false
-    end
-    return true
-  end
-
 end

app/controllers/application_controller.rb

@@ -1,7 +1,7 @@
 # The filters added to this controller will be run for all controllers in the application.
 # Likewise will all the methods added be available for all controllers.
 class ApplicationController < ActionController::Base
-#  require 'dnsbl_check'
+
   protect_forms_from_spam
   before_filter :connect_to_model, :check_authorization, :setup_url_generator, :set_content_type_header, :set_robots_metatag
   after_filter :remember_location, :teardown_url_generator
@@ -19,13 +19,31 @@ class ApplicationController < ActionController::Base
     Wiki.new
   end
 
+  helper_method :xhtml_enabled?, :html_ext, :darken
+
   protected
 
-  def check_authorization
-    if in_a_web? and authorization_needed? and not authorized?
-      redirect_to :controller => 'wiki', :action => 'login', :web => @web_name
-      return false
-    end
+  def xhtml_enabled?
+    in_a_web? and [:markdownMML, :markdownPNG, :markdown].include?(@web.markup)
+  end
+
+  def html_ext
+    if xhtml_enabled? && request.env.include?('HTTP_ACCEPT') &&
+           Mime::Type.parse(request.env["HTTP_ACCEPT"]).include?(Mime::XHTML)
+       'xhtml'
+    else
+      'html'
+    end       
+  end
+
+  def darken(s)
+     n=s.length/3
+     s.scan( %r(\w{#{n},#{n}}) ).collect {|a| (a.hex * 2/3).to_s(16).rjust(n,'0')}.join
+  end
+
+  def check_authorization    
+    redirect_to(:controller => 'wiki', :action => 'login',
+                :web => @web_name) if in_a_web? and authorization_needed? and not authorized?
   end
 
   def connect_to_model
@@ -35,10 +53,8 @@ class ApplicationController < ActionController::Base
     @author = cookies['author'] || 'AnonymousCoward'
     if @web_name
       @web = @wiki.webs[@web_name]
-      if @web.nil?
-        render(:status => 404, :text => "Unknown web '#{@web_name}'", :layout => 'error')
-        return false 
-      end
+      render(:status => 404, :text => "Unknown web '#{@web_name}'",
+             :layout => 'error') if @web.nil?
     end
   end
 
@@ -87,7 +103,8 @@ class ApplicationController < ActionController::Base
     original_options[:type] ||= (FILE_TYPES[File.extname(file_name)] or 'application/octet-stream')
     original_options[:disposition] ||= (DISPOSITION[original_options[:type]] or 'attachment')
     original_options[:stream] ||= false
-    original_options[:x_sendfile] = true if request.env.include?('HTTP_X_SENDFILE_TYPE') && request.remote_addr == LOCALHOST
+    original_options[:x_sendfile] = true if request.env.include?('HTTP_X_SENDFILE_TYPE') &&
+            ( request.remote_addr == LOCALHOST || defined?(PhusionPassenger) )
     original_options
   end
   
@@ -140,7 +157,7 @@ class ApplicationController < ActionController::Base
         <html xmlns="http://www.w3.org/1999/xhtml"><body>
           <h2>Internal Error</h2>
           <p>An application error occurred while processing your request.</p>
-          <!-- \n#{exception.to_s.is_utf8? ? exception.to_s.gsub!(/-{2,}/, '- -') : ''}\n#{exception.backtrace.join("\n")}\n -->
+          <!-- \n#{exception.to_s.purify.gsub!(/-{2,}/, '- -') }\n#{exception.backtrace.join("\n")}\n -->
         </body></html>
       EOL
   end
@@ -165,13 +182,11 @@ class ApplicationController < ActionController::Base
     end
   end
 
-  public
-
   def set_content_type_header
     response.charset = 'utf-8'
     if %w(atom_with_content atom_with_headlines).include?(action_name)
       response.content_type = Mime::ATOM
-    elsif %w(tex).include?(action_name)
+    elsif %w(tex tex_list).include?(action_name)
       response.content_type = Mime::TEXT
     elsif xhtml_enabled?
       if request.user_agent =~ /Validator/ or request.env.include?('HTTP_ACCEPT') &&
@@ -189,12 +204,6 @@ class ApplicationController < ActionController::Base
     end
   end
 
-  def xhtml_enabled?
-    in_a_web? and (@web.markup == :markdownMML or @web.markup == :markdownPNG or @web.markup == :markdown)
-  end
-
-  protected
-
   def set_robots_metatag
     if controller_name == 'wiki' and %w(show published s5).include? action_name and !(params[:mode] == 'diff')
       @robots_metatag_value = 'index,follow'
@@ -233,6 +242,16 @@ class ApplicationController < ActionController::Base
     (@web.published? and action_name == 's5')
   end
 
+  def is_post
+    unless (request.post? || Rails.env.test?)
+      layout = 'error'
+      layout = false if %w(tex tex_list).include?(action_name)
+      headers['Allow'] = 'POST'
+      render(:status => 405, :text => 'You must use an HTTP POST', :layout => layout)
+    end
+    return true
+  end
+
 end
 
 module Mime
@@ -257,10 +276,10 @@ end
 module Instiki
   module VERSION #:nodoc:
     MAJOR = 0
-    MINOR = 17
-    TINY  = 2 
+    MINOR = 19
+    TINY  = 3 
     SUFFIX = '(MML+)'
-    PRERELEASE =  false
+    PRERELEASE = false
     if PRERELEASE
        STRING = [MAJOR, MINOR].join('.') + PRERELEASE + SUFFIX
     else
@@ -268,3 +287,22 @@ module Instiki
     end
   end
 end
+
+# Monkey patch, to make Hash#key work in Ruby 1.8
+class Hash
+  alias_method(:key, :index) unless method_defined?(:key)
+end
+
+# Monkey patch, to ensure ActionCache doesn't muck with the content-type header.
+module ActionController #:nodoc:
+  module Caching
+    module Actions
+      class ActionCacheFilter
+        private
+          def set_content_type!(controller, extension)
+            return
+          end
+      end
+    end
+  end
+end

app/controllers/cache_sweeping_helper.rb

@@ -2,17 +2,17 @@ module CacheSweepingHelper
   
   def expire_cached_page(web, page_name)
     expire_action :controller => 'wiki', :web => web.address,
-        :action => %w(show published s5 tex print history), :id => page_name
+        :action => %w(show published s5 tex print history source), :id => page_name
     expire_action :controller => 'wiki', :web => web.address,
         :action => 'show', :id => page_name, :mode => 'diff'
   end
 
   def expire_cached_summary_pages(web)
-    categories = WikiReference.all(:conditions => "link_type = 'C'")
+    categories = WikiReference.list_categories(web)
     %w(recently_revised list).each do |action|
       expire_action :controller => 'wiki', :web => web.address, :action => action
       categories.each do |category|
-        expire_action :controller => 'wiki', :web => web.address, :action => action, :category => category.referenced_name
+        expire_action :controller => 'wiki', :web => web.address, :action => action, :category => category
       end
     end
 
@@ -26,12 +26,14 @@ module CacheSweepingHelper
   end
 
   def expire_cached_revisions(page)
-    page.revisions.count.times  do |i|
+    page.rev_ids.count.times  do |i|
       revno = i+1
       expire_action :controller => 'wiki', :web => page.web.address,
           :action => 'revision', :id => page.name, :rev => revno
       expire_action :controller => 'wiki', :web => page.web.address,
           :action => 'revision', :id => page.name, :rev => revno, :mode => 'diff'
+      expire_action :controller => 'wiki', :web => page.web.address,
+          :action => 'source', :id => page.name, :rev => revno
     end
   end
 

app/controllers/file_controller.rb

@@ -1,14 +1,14 @@
 # Controller responsible for serving files and pictures.
 
 require 'zip/zip'
-require 'stringsupport'
+require 'instiki_stringsupport'
 
 class FileController < ApplicationController
 
   layout 'default'
   
-  before_filter :dnsbl_check, :check_authorized
-  before_filter :check_allow_uploads, :except => [:file, :blahtex_png]
+  before_filter :check_authorized
+  before_filter :check_allow_uploads, :dnsbl_check, :except => [:file, :blahtex_png]
 
   def file
     @file_name = params['id']
@@ -18,6 +18,9 @@ class FileController < ApplicationController
       new_file = @web.wiki_files.create(params['file'])
       if new_file.valid?
         flash[:info] = "File '#{@file_name}' successfully uploaded"
+        WikiReference.pages_that_link_to_file(@web, @file_name).each do |page|
+          RevisionSweeper.expire_page(@web, page)
+        end
         redirect_to(params['referring_page'])
       else
         # pass the file with errors back into the form
@@ -88,50 +91,37 @@ class FileController < ApplicationController
   protected
   
   def check_authorized
-    if authorized? or @web.published?
-      return true
-    else
+    unless authorized? or @web.published?
       @hide_navigation  = true
       render(:status => 403, :text => 'This web is private', :layout => true)
-      return false
     end    
   end
 
   def check_allow_uploads
-    render(:status => 404, :text => "Web #{params['web'].inspect} not found", :layout => 'error') and return false unless @web
-    if @web.allow_uploads? and authorized?
-      return true
+    if @web
+      if @web.allow_uploads? and authorized?
+        return true
+      else
+        @hide_navigation  = true
+        render(:status => 403, :text => 'File uploads are blocked by the webmaster', :layout => true)
+        return false
+      end
     else
-      @hide_navigation  = true
-      render(:status => 403, :text => 'File uploads are blocked by the webmaster', :layout => true)
-      return false
+      render(:status => 404, :text => "Web #{params['web'].inspect} not found", :layout => 'error')
     end
   end
   
   private 
   
-  def is_post
-    unless (request.post? || ENV["RAILS_ENV"] == "test")
-      headers['Allow'] = 'POST'
-      render(:status => 405, :text => 'You must use an HTTP POST', :layout => 'error')
-      return false
-    end
-    return true
-  end
-
   def import_from_archive(archive)
     logger.info "Importing pages from #{archive}"
     zip = Zip::ZipInputStream.open(archive)
     while (entry = zip.get_next_entry) do
       ext_length = File.extname(entry.name).length
-      page_name = entry.name[0..-(ext_length + 1)]
-      page_content = entry.get_input_stream.read
+      page_name = entry.name[0..-(ext_length + 1)].purify
+      page_content = entry.get_input_stream.read.purify
       logger.info "Processing page '#{page_name}'"
       begin
-        if !page_content.is_utf8?
-          logger.info "Page '#{page_name}' contains non-utf8 character data. Skipping."
-          next
-        end
         existing_page = @wiki.read_page(@web.address, page_name)
         if existing_page
           if existing_page.content == page_content

app/controllers/revision_sweeper.rb

@@ -7,7 +7,8 @@ class RevisionSweeper < ActionController::Caching::Sweeper
   observe Revision, Page
   
   def before_save(record)
-    if record.is_a?(Revision) 
+    if record.is_a?(Revision)
+      expire_cached_page(record.page.web, record.page.name) 
       expire_cached_revisions(record.page)
     end
   end
@@ -17,18 +18,30 @@ class RevisionSweeper < ActionController::Caching::Sweeper
       expire_caches(record.page)
     end
   end
-  
+
+  def after_create(record)
+    if record.is_a?(Page)
+       WikiReference.pages_that_reference(record.web, record.name).each do |page_name|
+         expire_cached_page(record.web, page_name)
+      end
+    end
+  end
+
   def after_delete(record)
     if record.is_a?(Page)
       expire_caches(record)
     end
   end
   
+  def self.expire_page(web, page_name)
+    new.expire_cached_page(web, page_name)
+  end
+
   private
   
   def expire_caches(page)
     expire_cached_summary_pages(page.web)
-    pages_to_expire = ([page.name] + WikiReference.pages_that_reference(page.web, page.name) +
+    pages_to_expire = ([page.name] + 
        WikiReference.pages_redirected_to(page.web, page.name) +
        WikiReference.pages_that_include(page.web, page.name)).uniq
     pages_to_expire.each { |page_name| expire_cached_page(page.web, page_name) }

app/controllers/web_sweeper.rb

@@ -9,7 +9,7 @@ class WebSweeper < ActionController::Caching::Sweeper
   def after_save(record)
     if record.is_a?(Web)
       web = record
-      web.pages.each { |page| expire_cached_page(web, page.name) }
+      web.pages.find_each { |page| expire_cached_page(web, page.name) }
       expire_cached_summary_pages(web)
     elsif record.is_a?(WikiFile)
       record.web.pages_that_link_to_file(record.file_name).each do |page|

app/controllers/wiki_controller.rb

@@ -1,18 +1,19 @@
 require 'fileutils'
 require 'maruku'
+require 'maruku/ext/math'
 require 'zip/zip'
-require 'stringsupport'
+require 'instiki_stringsupport'
 require 'resolv'
 
 class WikiController < ApplicationController
 
   before_filter :load_page
   before_filter :dnsbl_check, :only => [:edit, :new, :save, :export_html, :export_markup]
-  caches_action :show, :published, :authors, :tex, :s5, :print, :recently_revised, :list, :file_list,
+  caches_action :show, :published, :authors, :tex, :s5, :print, :recently_revised, :list, :file_list, :source,
         :history, :revision, :atom_with_content, :atom_with_headlines, :if => Proc.new { |c| c.send(:do_caching?) }
   cache_sweeper :revision_sweeper
 
-  layout 'default', :except => [:atom_with_content, :atom_with_headlines, :atom, :tex, :s5, :export_html]
+  layout 'default', :except => [:atom_with_content, :atom_with_headlines, :atom, :source, :tex, :s5, :export_html]
 
   def index
     if @web_name
@@ -67,42 +68,65 @@ class WikiController < ApplicationController
   end
   
   def export_html
-    stylesheet = File.read(File.join(RAILS_ROOT, 'public', 'stylesheets', 'instiki.css'))
     export_pages_as_zip(html_ext) do |page| 
-
-      renderer = PageRenderer.new(page.revisions.last)
+      renderer = PageRenderer.new(page.current_revision)
       rendered_page = <<-EOL
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1 plus MathML 2.0 plus SVG 1.1//EN" "http://www.w3.org/2002/04/xhtml-math-svg/xhtml-math-svg-flat.dtd" >
-        <html xmlns="http://www.w3.org/1999/xhtml">
-        <head>
-          <title>#{page.plain_name} in #{@web.name}</title>
-          <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+  <title>#{page.plain_name} in #{@web.name}</title>
+  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
   
-          <style type="text/css">
-            h1#pageName, .newWikiWord a, a.existingWikiWord, .newWikiWord a:hover { 
-              color: ##{@web ? @web.color : "393" }; 
-            }
-            .newWikiWord { background-color: white; font-style: italic; }
-            #{stylesheet}
-          </style>
-          <style type="text/css">
-            #{@web.additional_style}
-          </style>
-        </head>
-        <body>
-          <h1 id="pageName">
-            <span class="webName">#{@web.name}</span><br />
-            #{page.plain_name}    
-          </h1>
-          #{renderer.display_content_for_export}
-          <div class="byline">
-            #{page.revisions? ? "Revised" : "Created" } on #{ page.revised_at.strftime('%B %d, %Y %H:%M:%S') }
-            by
-            #{ UrlGenerator.new(self).make_link(@web, page.author.name, @web, nil, { :mode => :export }) }
-          </div>
-        </body>
-        </html>
-      EOL
+  <script src="public/javascripts/page_helper.js" type="text/javascript"></script> 
+  <link href="public/stylesheets/instiki.css" media="all" rel="stylesheet" type="text/css" />
+  <link href="public/stylesheets/syntax.css" media="all" rel="stylesheet" type="text/css" />
+  <style type="text/css">
+    h1#pageName, div.info, .newWikiWord a, a.existingWikiWord, .newWikiWord a:hover, [actiontype="toggle"]:hover, #TextileHelp h3 { 
+      color: ##{@web ? @web.color : "393"}; 
+    }
+    a:visited.existingWikiWord {
+      color: ##{darken(@web ? @web.color : "393")};
+    }   
+  </style>
+  
+  <style type="text/css"><!--/*--><![CDATA[/*><!--*/    
+    #{@web ? @web.additional_style : ''}
+  /*]]>*/--></style>
+  <script src="public/javascripts/prototype.js" type="text/javascript"></script>
+  <script src="public/javascripts/effects.js" type="text/javascript"></script>
+  <script src="public/javascripts/dragdrop.js" type="text/javascript"></script>
+  <script src="public/javascripts/controls.js" type="text/javascript"></script>
+  <script src="public/javascripts/application.js" type="text/javascript"></script>
+
+</head>
+<body>
+ <div id="Container">
+  <div id="Content">
+  <h1 id="pageName">
+  #{xhtml_enabled? ? %{<span id="svg_logo"><svg version="1.1" width="100%" height="100%" viewBox='0 -1 180 198' xmlns='http://www.w3.org/2000/svg'>
+      <path id="svg_logo_path" fill="##{@web ? @web.color : "393"}" stroke-width='0.5' stroke='#000' d='
+        M170,60c4,11-1,20-12,25c-9,4-25,3-20,15c5,5,15,0,24,1c11,1,21,11,14,21c-10,15-35,6-48-1c-5-3-27-23-32-10c-1,13,15,10,22,16
+        c11,4,24,14,34,20c12,10,7,25-9,23c-11-1-22-9-30-16c-5-5-13-18-21-9c-2,6,2,11,5,14c9,9,22,14,22,31c-2,8-12,8-18,4c-4-3-9-8-11-13
+        c-3-6-5-18-12-18c-14-1-5,28-18,30c-9,2-13-9-12-16c1-14,12-24,21-31c5-4,17-13,10-20c-9-10-19,12-23,16c-7,7-17,16-31,15
+        c-9-1-18-9-11-17c5-7,14-4,23-6c6-1,15-8,8-15c-5-6-57,2-42-24c7-12,51,4,61,6c6,1,17,4,18-4c2-11-12-7-21-8c-21-2-49-14-49-34
+        c0-5,3-11,8-11C31,42,34,65,42,67c6,1,9-3,8-9C49,49,38,40,40,25c1-5,4-15,13-14c10,2,11,18,13,29c1,8,0,24,7,28c15,0,5-22,4-30
+        C74,23,78,7,87,1c8-4,14,1,16,9c2,11-8,21-2,30c8,2,11-6,14-12c9-14,36-18,30,5c-3,9-12,19-21,24c-6,4-22,10-23,19c-2,14,15,2,18-2
+        c9-9,20-18,33-22C159,52,166,54,170,60' />
+    </svg></span>} : ''}
+  <span class="webName">#{@web.name}</span><br />
+  #{page.plain_name}    
+  </h1>
+#{renderer.display_content_for_export}
+  <div class="byline">
+  #{page.revisions? ? "Revised" : "Created" } on #{ page.revised_at.strftime('%B %d, %Y %H:%M:%S') }
+  by
+  #{ UrlGenerator.new(self).make_link(@web, page.author.name, @web, nil, { :mode => :export }) }
+  </div>
+  </div>
+ </div>
+</body>
+</html>
+EOL
       rendered_page
     end
   end
@@ -160,10 +184,34 @@ class WikiController < ApplicationController
   def atom_with_headlines
     render_atom(hide_description = true)
   end
+  
+  def tex_list
+    return unless is_post
+    if [:markdownMML, :markdownPNG, :markdown].include?(@web.markup)
+      @tex_content = ''
+      # Ruby 1.9.x has ordered hashes; 1.8.x doesn't. So let's just parse the query ourselves.
+      ordered_params = ActiveSupport::OrderedHash[*request.raw_post.split('&').collect {|k_v| k_v.split('=').collect {|x| CGI::unescape(x)}}.flatten]
+      ordered_params.each do |name, p|
+        if p == 'tex' && @web.has_page?(name)
+          @tex_content << "\\section*\{#{Maruku.new(name).to_latex.strip}\}\n\n"
+          @tex_content << Maruku.new(@web.page(name).content).to_latex
+        end
+      end
+    else
+      @tex_content = 'TeX export only supported with the Markdown text filters.'
+    end
+    if @tex_content == ''
+      flash[:error] = "You didn't select any pages to export."
+      redirect_to :back
+      return
+    end
+    expire_action :controller => 'wiki', :web => @web.address, :action => 'list', :category => params['category']
+    render(:layout => 'tex')
+  end
+
 
   def search
-    @query = params['query']
-    render(:text => "Your query string was not valid utf-8", :layout => 'error', :status => 400) and return unless @query.is_utf8?
+    @query = params['query'] ? params['query'].purify : ''
     @title_results = @web.select { |page| page.name =~ /#{@query}/i }.sort
     @results = @web.select { |page| page.content =~ /#{@query}/i }.sort
     all_pages_found = (@results + @title_results).uniq
@@ -180,7 +228,7 @@ class WikiController < ApplicationController
   end
 
   def edit
-    if @page.nil? or not @page_name.is_utf8?
+    if @page.nil?
       redirect_home
     elsif @page.locked?(Time.now) and not params['break_lock']
       redirect_to :web => @web_name, :action => 'locked', :id => @page_name
@@ -190,12 +238,10 @@ class WikiController < ApplicationController
   end
   
   def locked
-    render(:text => 'Page name is not valid utf-8.', :status => 400, :layout => 'error') unless @page_name.is_utf8? 
     # to template
   end
   
   def new
-    render(:text => 'Page name is not valid utf-8.', :status => 400, :layout => 'error') unless @page_name.is_utf8? 
     # to template
   end
 
@@ -216,7 +262,7 @@ class WikiController < ApplicationController
       redirect_home
     end
     @link_mode ||= :show
-    @renderer = PageRenderer.new(@page.revisions.last)
+    @renderer = PageRenderer.new(@page.current_revision)
     # to template
   end
 
@@ -230,7 +276,7 @@ class WikiController < ApplicationController
     @page ||= wiki.read_page(@web_name, @page_name)
     @link_mode ||= :publish
     if @page
-       @renderer = PageRenderer.new(@page.revisions.last)
+       @renderer = PageRenderer.new(@page.current_revision)
     else
       real_page = WikiReference.page_that_redirects_for(@web, @page_name)
         if real_page
@@ -258,34 +304,22 @@ class WikiController < ApplicationController
   end
 
   def save
-    render(:status => 404, :text => 'Undefined page name', :layout => 'error') and return if @page_name.nil? or not @page_name.is_utf8?
-    unless (request.post? || ENV["RAILS_ENV"] == "test")
-      headers['Allow'] = 'POST'
-      render(:status => 405, :text => 'You must use an HTTP POST', :layout => 'error')
-      return
-    end
-    author_name = params['author']
+    render(:status => 404, :text => 'Undefined page name', :layout => 'error') and return if @page_name.nil?
+    return unless is_post
+    author_name = params['author'].purify
     author_name = 'AnonymousCoward' if author_name =~ /^\s*$/
     
     begin
-      raise Instiki::ValidationError.new('Your name was not valid utf-8') unless author_name.is_utf8?
-      raise Instiki::ValidationError.new('Your name cannot contain a "."') if author_name.include? '.'
-      cookies['author'] = { :value => author_name, :expires => Time.utc(2030) }
-      the_content = params['content']
+      the_content = params['content'].purify
+      prev_content = ''
       filter_spam(the_content)
-      unless the_content.is_utf8?
-        if @page
-          the_content = @page.content
-        else
-          the_content = ''
-        end 
-        raise Instiki::ValidationError.new('Your content was not valid utf-8.')
-      end
+      cookies['author'] = { :value => author_name.dup.as_bytes, :expires => Time.utc(2030) }
       if @page
-        new_name = params['new_name'] || @page_name
-        raise Instiki::ValidationError.new('Your new title was not valid utf-8.') unless new_name.is_utf8?
-        raise Instiki::ValidationError.new('Your new title cannot contain a "."') if new_name.include? '.'
-        raise Instiki::ValidationError.new('A page named "' + new_name.escapeHTML + '" already exists.') if @page_name != new_name && @web.has_page?(new_name)
+        new_name = params['new_name'] ? params['new_name'].purify : @page_name
+        new_name = @page_name if new_name.empty?
+        prev_content = @page.current_revision.content
+        raise Instiki::ValidationError.new('A page named "' + new_name.escapeHTML + '" already exists.') if
+            @page_name != new_name && @web.has_page?(new_name)
         wiki.revise_page(@web_name, @page_name, new_name, the_content, Time.now, 
             Author.new(author_name, remote_ip), PageRenderer.new)
         @page.unlock
@@ -298,11 +332,15 @@ class WikiController < ApplicationController
     rescue Instiki::ValidationError => e
       flash[:error] = e.to_s
       logger.error e
+      param_hash = {:web => @web_name, :id => @page_name}
+      # Work around Rails bug: flash will not display if query string is longer than 10192 bytes
+      param_hash.update( :content => the_content ) if the_content && 
+         CGI::escape(the_content).length < 10183 && the_content != prev_content
       if @page
         @page.unlock
-        redirect_to :action => 'edit', :web => @web_name, :id => @page_name, :content => the_content
+        redirect_to param_hash.update( :action => 'edit' )
       else
-        redirect_to :action => 'new', :web => @web_name, :id => @page_name, :content => the_content
+        redirect_to param_hash.update( :action => 'new' )
       end
     end
   end
@@ -310,7 +348,7 @@ class WikiController < ApplicationController
   def show
     if @page
       begin
-        @renderer = PageRenderer.new(@page.revisions.last)
+        @renderer = PageRenderer.new(@page.current_revision)
         @show_diff = (params[:mode] == 'diff')
         render :action => 'page'
       # TODO this rescue should differentiate between errors due to rendering and errors in 
@@ -325,7 +363,7 @@ class WikiController < ApplicationController
         end
       end
     else
-      if not @page_name.nil? and @page_name.is_utf8? and not @page_name.empty?
+      if not @page_name.nil? and not @page_name.empty?
         real_page = WikiReference.page_that_redirects_for(@web, @page_name)
         if real_page
           flash[:info] = "Redirected from \"#{@page_name}\"."
@@ -345,8 +383,8 @@ class WikiController < ApplicationController
     if @page
       @revisions_by_day = Hash.new { |h, day| h[day] = [] }
       @revision_numbers = Hash.new { |h, id| h[id] = [] }
-      revision_number = @page.revisions.size
-      @page.revisions.reverse.each do |rev|
+      revision_number = @page.rev_ids.size
+      @page.rev_ids.reverse.each do |rev|
         day = Date.new(rev.revised_at.year, rev.revised_at.month, rev.revised_at.day)
         @revisions_by_day[day] << rev
         @revision_numbers[rev.id] = revision_number
@@ -354,7 +392,7 @@ class WikiController < ApplicationController
       end
       render :action => 'history'
     else
-      if not @page_name.nil? and @page_name.is_utf8? and not @page_name.empty?
+      if not @page_name.nil? and not @page_name.empty?
         redirect_to :web => @web_name, :action => 'new', :id => @page_name
       else
         render :text => 'Page name is not specified', :status => 404, :layout => 'error'
@@ -362,17 +400,22 @@ class WikiController < ApplicationController
     end
   end
 
+  def source
+    @revision = @page.revisions[params['rev'].to_i - 1] if params['rev']
+  end
+
   def tex
-    if @web.markup == :markdownMML or @web.markup == :markdownPNG or @web.markup == :markdown
+    if [:markdownMML, :markdownPNG, :markdown].include?(@web.markup)
       @tex_content = Maruku.new(@page.content).to_latex
     else
       @tex_content = 'TeX export only supported with the Markdown text filters.'
     end
+    render(:layout => 'tex')
   end
 
   def s5
-    if @web.markup == :markdownMML || @web.markup == :markdownPNG || @web.markup == :markdown
-      my_rendered = PageRenderer.new(@page.revisions.last)
+    if [:markdownMML, :markdownPNG, :markdown].include?(@web.markup)
+      my_rendered = PageRenderer.new(@page.current_revision)
       @s5_content = my_rendered.display_s5
       @s5_theme = my_rendered.s5_theme
     else
@@ -381,15 +424,6 @@ class WikiController < ApplicationController
     end
   end
 
-  def html_ext
-    if xhtml_enabled? && request.env.include?('HTTP_ACCEPT') &&
-           Mime::Type.parse(request.env["HTTP_ACCEPT"]).include?(Mime::XHTML)
-       'xhtml'
-    else
-      'html'
-    end       
-  end
-
   protected
 
   def do_caching?
@@ -397,7 +431,7 @@ class WikiController < ApplicationController
   end
   
   def load_page
-    @page_name = params['id']
+    @page_name = params['id'] ? params['id'].purify : nil
     @page = @wiki.read_page(@web_name, @page_name) if @page_name
   end
 
@@ -428,22 +462,32 @@ class WikiController < ApplicationController
     
     file_prefix = "#{@web.address}-#{file_type}-"
     timestamp = @web.revised_at.strftime('%Y-%m-%d-%H-%M-%S')
-    file_path = File.join(@wiki.storage_path, file_prefix + timestamp + '.zip')
+    file_path = @wiki.storage_path.join(file_prefix + timestamp + '.zip')
     tmp_path = "#{file_path}.tmp"
 
-    Zip::ZipOutputStream.open(tmp_path) do |zip_out|
+    Zip::ZipFile.open(tmp_path, Zip::ZipFile::CREATE) do |zip_out|
       @web.select.by_name.each do |page|
-        zip_out.put_next_entry("#{CGI.escape(page.name)}.#{file_type}")
-        zip_out.puts(block.call(page))
+        zip_out.get_output_stream("#{CGI.escape(page.name)}.#{file_type}") do |f|
+          f.puts(block.call(page))
+        end
       end
-      # add an index file, if exporting to HTML
+      # add an index file, and the stylesheet and javascript directories, if exporting to HTML
       if file_type.to_s.downcase == html_ext
-        zip_out.put_next_entry "index.#{html_ext}"
-        zip_out.puts "<html xmlns='http://www.w3.org/1999/xhtml'><head>" +
-            "<META HTTP-EQUIV=\"Refresh\" CONTENT=\"0;URL=HomePage.#{file_type}\"></head></html>"
+        zip_out.get_output_stream("index.#{html_ext}") do |f|
+          f.puts "<html xmlns='http://www.w3.org/1999/xhtml'><head>" +
+            "<meta http-equiv=\"Refresh\" content=\"0;URL=HomePage.#{html_ext}\" /></head></html>"
+        end
+        dir = Rails.root.join('public')
+        Dir["#{dir}/{images,javascripts,s5,stylesheets}/**/*"].each do |f|
+          zip_out.add "public#{f.sub(dir.to_s,'')}", f
+        end
+      end
+      files = @web.files_path
+      Dir["#{files}/**/*"].each do |f|
+        zip_out.add "files#{f.sub(files.to_s,'')}", f
       end
     end
-    FileUtils.rm_rf(Dir[File.join(@wiki.storage_path, file_prefix + '*.zip')])
+    FileUtils.rm_rf(Dir[@wiki.storage_path.join(file_prefix + '*.zip').to_s])
     FileUtils.mv(tmp_path, file_path)
     send_file file_path
   end
@@ -462,7 +506,7 @@ class WikiController < ApplicationController
     if params['rev']
       @revision_number = params['rev'].to_i
     else
-      @revision_number = @page.revisions.size
+      @revision_number = @page.rev_ids.size
     end
     @revision = @page.revisions[@revision_number - 1]
   end
@@ -508,11 +552,7 @@ class WikiController < ApplicationController
   def rss_with_content_allowed?
     @web.password.nil? or @web.published?
   end
-  
-  def truncate(text, length = 30, truncate_string = '...')
-    if text.length > length then text[0..(length - 3)] + truncate_string else text end
-  end
-  
+
   def filter_spam(content)
     @@spam_patterns ||= load_spam_patterns
     @@spam_patterns.each do |pattern| 
@@ -521,9 +561,9 @@ class WikiController < ApplicationController
   end
 
   def load_spam_patterns
-    spam_patterns_file = "#{RAILS_ROOT}/config/spam_patterns.txt"
+    spam_patterns_file = Rails.root.join('config', 'spam_patterns.txt')
     if File.exists?(spam_patterns_file)
-      File.readlines(spam_patterns_file).inject([]) { |patterns, line| patterns << Regexp.new(line.chomp, Regexp::IGNORECASE) } 
+      spam_patterns_file.readlines.inject([]) { |patterns, line| patterns << Regexp.new(line.chomp, Regexp::IGNORECASE) } 
     else
       []
     end

app/helpers/application_helper.rb

@@ -1,5 +1,6 @@
 # The methods added to this helper will be available to all templates in the application.
 module ApplicationHelper
+require 'instiki_stringsupport'
 
   # Accepts a container (hash, array, enumerable, your type) and returns a string of option tags. Given a container 
   # where the elements respond to first and last (such as a two-element array), the "lasts" serve as option values and
@@ -30,7 +31,7 @@ module ApplicationHelper
       end
     end
     
-    html_options.join("\n")
+    html_options.join("\n").html_safe
   end
 
   # Creates a hyperlink to a Wiki page, without checking if the page exists or not
@@ -38,39 +39,39 @@ module ApplicationHelper
     link_to(
         text || page.plain_name, 
         {:web => @web.address, :action => 'show', :id => page.name, :only_path => true},
-        html_options)
+        html_options).html_safe
   end
   
   # Creates a hyperlink to a Wiki page, or to a "new page" form if the page doesn't exist yet
   def link_to_page(page_name, web = @web, text = nil, options = {})
     raise 'Web not defined' if web.nil?
     UrlGenerator.new(@controller).make_link(@web, page_name, web, text, 
-        options.merge(:base_url => "#{base_url}/#{web.address}"))
+        options.merge(:base_url => "#{base_url}/#{web.address}")).html_safe
   end
 
   def author_link(page, options = {})
-    UrlGenerator.new(@controller).make_link(@web, page.author.name, page.web, nil, options)
+    UrlGenerator.new(@controller).make_link(@web, page.author.name, page.web, nil, options).purify.html_safe
   end
 
   # Create a hyperlink to a particular revision of a Wiki page
   def link_to_revision(page, revision_number, text = nil, mode = nil, html_options = {})
-    revision_number == page.revisions.size ?
+    revision_number == page.rev_ids.size ?
       link_to(
         text || page.plain_name,
             {:web => @web.address, :action => 'show', :id => page.name,
-               :mode => mode}, html_options) :
+               :mode => mode}, html_options).html_safe :
       link_to(
-        text || page.plain_name + "(rev # #{revision_number})",
+        text || page.plain_name + "(rev # #{revision_number})".html_safe,
             {:web => @web.address, :action => 'revision', :id => page.name,
-              :rev => revision_number, :mode => mode}, html_options)
+              :rev => revision_number, :mode => mode}, html_options).html_safe
   end
 
   # Create a hyperlink to the history of a particular Wiki page
   def link_to_history(page, text = nil, html_options = {})
     link_to(
-        text || page.plain_name + "(history)",
+        text || page.plain_name + "(history)".html_safe,
             {:web => @web.address, :action => 'history', :id => page.name},
-            html_options)
+            html_options).html_safe
   end
 
   def base_url
@@ -83,19 +84,19 @@ module ApplicationHelper
     if @categories.empty?
       ''
     else 
-      "<div id=\"categories\">\n" +
+      ("<div id=\"categories\">\n" +
       '<strong>Categories</strong>:' +
       '[' + link_to_unless_current('Any', :web => @web.address, :action => self.action_name, :category => nil) + "]\n" +
       @categories.map { |c| 
-        link_to_unless_current(c, :web => @web.address, :action => self.action_name, :category => c)
+        link_to_unless_current(c.html_safe, :web => @web.address, :action => self.action_name, :category => c)
       }.join(', ') + "\n" +
-      '</div>'
+      '</div>').html_safe
     end
   end
 
   # Performs HTML escaping on text, but keeps linefeeds intact (by replacing them with <br/>)
   def escape_preserving_linefeeds(text)
-    h(text).gsub(/\n/, '<br/>')
+    h(text).gsub(/\n/, '<br/>').as_utf8.html_safe
   end
 
   def format_date(date, include_time = true)
@@ -108,7 +109,22 @@ module ApplicationHelper
   end
 
   def rendered_content(page)
-    PageRenderer.new(page.revisions.last).display_content
+    PageRenderer.new(page.current_revision).display_content
+  end
+
+  def truncate(text, *args)
+    options = args.extract_options!
+    options.reverse_merge!(:length => 30, :omission => "...")
+    return text.html_safe if text.num_chars <= options[:length]
+    len = options[:length] - options[:omission].as_utf8.num_chars
+    t = ''
+    text.split.collect do |word|
+      if t.num_chars + word.num_chars <= len
+        t << word + ' '
+      else 
+        return (t.chop + options[:omission]).html_safe
+      end
+    end
   end
 
 end

app/helpers/wiki_helper.rb

@@ -6,7 +6,7 @@ module WikiHelper
     menu << back_for_revision if @revision_number > 1
     menu << current_revision
     menu << see_or_hide_changes_for_revision if @revision_number > 1
-    menu << history if @page.revisions.size > 1
+    menu << history if @page.rev_ids.size > 1
     menu << rollback
     menu
   end
@@ -15,11 +15,11 @@ module WikiHelper
     menu = []
     menu << edit_page
     menu << edit_web if @page.name == "HomePage"
-    if @page.revisions.size > 1
+    if @page.rev_ids.size > 1
       menu << back_for_page
       menu << see_or_hide_changes_for_page
     end
-    menu << history if @page.revisions.size > 1
+    menu << history if @page.rev_ids.size > 1
     menu
   end
 
@@ -40,15 +40,15 @@ module WikiHelper
   end
             
   def forward
-    if @revision_number < @page.revisions.size - 1
+    if @revision_number < @page.rev_ids.size - 1
       link_to('Forward in time', 
           {:web => @web.address, :action => 'revision', :id => @page.name, :rev => @revision_number + 1},
           {:class => 'navlink', :accesskey => 'F', :id => 'to_next_revision', :rel => 'nofollow'}) + 
-          " <span class='revisions'>(#{@revision.page.revisions.size - @revision_number} more)</span> "
+          " <span class='revisions'>(#{@revision.page.rev_ids.size - @revision_number} more)</span> ".html_safe
     else
         link_to('Forward in time', {:web => @web.address, :action => 'show', :id => @page.name},
             {:class => 'navlink', :accesskey => 'F', :id => 'to_next_revision', :rel => 'nofollow'}) +
-            " <span class='revisions'>(to current)</span>"
+            " <span class='revisions'>(to current)</span>".html_safe
     end
   end
     
@@ -56,15 +56,15 @@ module WikiHelper
     link_to('Back in time',
         {:web => @web.address, :action => 'revision', :id => @page.name, :rev => @revision_number - 1},
         {:class => 'navlink', :id => 'to_previous_revision', :rel => 'nofollow'}) + 
-        " <span class='revisions'>(#{@revision_number - 1} more)</span>"
+        " <span class='revisions'>(#{@revision_number - 1} more)</span>".html_safe
   end
 
   def back_for_page
     link_to('Back in time', 
         {:web => @web.address, :action => 'revision', :id => @page.name, 
-        :rev => @page.revisions.size - 1},
+        :rev => @page.rev_ids.size - 1},
         {:class => 'navlink', :accesskey => 'B', :id => 'to_previous_revision', :rel => 'nofollow'}) +
-        " <span class='revisions'>(#{@page.revisions.size - 1} #{@page.revisions.size - 1 == 1 ? 'revision' : 'revisions'})</span>"
+        " <span class='revisions'>(#{@page.rev_ids.size - 1} #{@page.rev_ids.size - 1 == 1 ? 'revision' : 'revisions'})</span>".html_safe
   end
   
   def current_revision
@@ -91,6 +91,4 @@ module WikiHelper
         {:class => 'navlink', :id => 'rollback', :rel => 'nofollow'})
   end
 
-  
-
 end

app/metal/itex.rb

@@ -0,0 +1,75 @@
+ # Allow the metal piece to run in isolation
+require(File.dirname(__FILE__) + "/../../config/environment") unless defined?(Rails)
+
+require 'instiki_stringsupport'
+
+class Itex
+  def self.call(env)
+    if env["PATH_INFO"] =~ /^\/itex/
+      [200, {"Content-Type" => "application/xml"}, [response(env)]]
+    else
+      [404, {"Content-Type" => "text/html"}, ["Not Found"]]
+    end
+  end
+  
+  private
+
+  # plugable XML parser; falls back to REXML
+  begin
+    require 'nokogiri'
+    def self.xmlparse(text)
+      Nokogiri::XML(text) { |config| config.strict }
+    end
+  rescue LoadError
+    require 'rexml/document'
+    def self.xmlparse(text)
+      REXML::Document.new(text)
+    end
+  end
+
+  #error message to return
+  def self.error(str)
+    "<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><merror><mtext>" +
+       str + "</mtext></merror></math>"
+  end
+
+  # itex2MML parser
+  begin
+    require 'itextomml'
+    def self.parse_itex(tex, filter)
+      Itex2MML::Parser.new.send(filter, tex).to_utf8
+    end
+  rescue LoadError
+    def self.parse_itex(tex, filter)
+      error("Please install the itex2MML Ruby bindings.")
+    end  
+  end
+
+  # the actual response
+  def self.response(env)
+    params = Rack::Request.new(env).params
+    tex = (params['tex'] || '').purify.strip
+    case params['display']
+      when 'block'
+        filter = :block_filter
+      else
+         filter = :inline_filter
+    end
+    return "<math xmlns='http://www.w3.org/1998/Math/MathML' display='" +
+        filter.to_s[/(.*?)_filter/] + "'/>" if tex == ''
+    begin
+      doc = parse_itex(tex, filter)
+      # make sure the result is well-formed, before sending it off
+      begin
+        xmlparse(doc)
+      rescue
+        return error("Ill-formed XML.")
+      end
+      return doc
+    rescue Itex2MML::Error => e
+      error(e.to_s)
+    rescue
+      error("Unknown Error")
+    end
+  end
+end

app/models/author.rb

@@ -3,7 +3,7 @@ class Author < String
   attr_reader :name
   def initialize(name, ip = nil) 
     @ip = ip
-    super(name)
+    super(name.as_utf8)
   end
 
   def name=(value)

app/models/page.rb

@@ -1,11 +1,17 @@
 class Page < ActiveRecord::Base
   belongs_to :web
   has_many :revisions, :order => 'id', :dependent => :destroy
+  #In many cases, we don't need to instantiate the full revisions (with all that textual data)
+  has_many :rev_ids, :order => 'id', :class_name => 'Revision', :select => 'id, revised_at, page_id, author, ip'
   has_many :wiki_references, :order => 'referenced_name'
   has_one :current_revision, :class_name => 'Revision', :order => 'id DESC'
 
+  def name
+    read_attribute(:name).as_utf8
+  end
+
   def revise(content, name, time, author, renderer)
-    revisions_size = new_record? ? 0 : revisions.size
+    revisions_size = new_record? ? 0 : rev_ids.size
     if (revisions_size > 0) and content == current_revision.content and name == self.name
       raise Instiki::ValidationError.new(
           "You have tried to save page '#{name}' without changing its content")
@@ -42,11 +48,11 @@ class Page < ActiveRecord::Base
   end
 
   def revisions?
-    revisions.size > 1
+    rev_ids.size > 1
   end
 
   def previous_revision(revision)
-    revision_index = revisions.each_with_index do |rev, index| 
+    revision_index = rev_ids.each_with_index do |rev, index| 
       if rev.id == revision.id 
         break index 
       else
@@ -68,6 +74,10 @@ class Page < ActiveRecord::Base
     wiki_references.select { |ref| ref.wiki_word? }.map { |ref| ref.referenced_name }
   end
 
+  def categories
+    wiki_references.select { |ref| ref.category? }.map { |ref| ref.referenced_name }
+  end
+
   def linked_from
     web.select.pages_that_link_to(name)
   end
@@ -82,7 +92,7 @@ class Page < ActiveRecord::Base
 
   # Returns the original wiki-word name as separate words, so "MyPage" becomes "My Page".
   def plain_name
-    web.brackets_only? ? CGI.escapeHTML(name) : CGI.escapeHTML(WikiWords.separate(name))
+    web.brackets_only? ? name.escapeHTML.html_safe : WikiWords.separate(name).escapeHTML.html_safe
   end
 
   LOCKING_PERIOD = 30.minutes
@@ -104,7 +114,7 @@ class Page < ActiveRecord::Base
   end
 
   def to_param
-    name
+    name.as_utf8
   end
 
   private

app/models/revision.rb

@@ -1,4 +1,8 @@
 class Revision < ActiveRecord::Base
   belongs_to :page
   composed_of :author, :mapping => [ %w(author name), %w(ip ip) ]
+
+  def content
+    read_attribute(:content).as_utf8
+  end
 end

app/models/web.rb

@@ -1,48 +1,75 @@
+require 'instiki_stringsupport'
+
 class Web < ActiveRecord::Base
-  has_many :pages, :dependent => :destroy
+  ## Associations
+
+  has_many :pages,      :dependent => :destroy
   has_many :wiki_files, :dependent => :destroy
 
+  has_many :revisions,  :through => :pages
+
+  ## Hooks
+
+  before_save :sanitize_markup
+  after_save :create_files_directory
+
+  before_validation :validate_address
+
+  ## Validations
+
+  validates_uniqueness_of :address, :message => 'already exists'
+  validates_length_of :color, :in => 3..6
+
+  ## Methods
+
+  # @return [Wiki] a new Wiki instance
   def wiki
     Wiki.new
   end
-  
+
   def settings_changed?(markup, safe_mode, brackets_only)
     self.markup != markup || 
     self.safe_mode != safe_mode || 
     self.brackets_only != brackets_only
   end
-  
+
   def add_page(name, content, time, author, renderer)
-    page = page(name) || Page.new(:web => self, :name => name)
+    page = page(name) || pages.build(:name => name)
     page.revise(content, name, time, author, renderer)
   end
-  
+
+  # @return [Array<String>] a collection of all the names of the authors that 
+  #   have ever contributed to the pages for this Web
   def authors
-    connection.select_all(
-        'SELECT DISTINCT r.author AS author ' + 
-        'FROM revisions r ' +
-        'JOIN pages p ON p.id = r.page_id ' +
-        'WHERE p.web_id = ' + self.id.to_s +
-        ' ORDER by 1 '
-        ).collect { |row| row['author'] }        
+    revisions.all(
+      :select => "DISTINCT revisions.author",
+      :order  => "1"
+    ).collect(&:author)
   end
 
   def categories
     select.map { |page| page.categories }.flatten.uniq.sort
   end
 
+  # @param [String] name the name of some associated Page record to find
+  # @return [Page, nil] the associated Page record, or +nil+ if no record is 
+  #   found with the provided name
   def page(name)
-    pages.first(:conditions => ['name = ?', name])
-  end
-  
-  def last_page
-    return Page.first(:order => 'id desc', :conditions => ['web_id = ?', self.id])
+    pages.find_by_name(name)
   end
 
-  def has_page?(name)
-    Page.count(:conditions => ['web_id = ? AND name = ?', id, name]) > 0
+  # @return [Page] the last associated Page record
+  def last_page
+    pages.last
   end
-  
+
+  # @param [String] name the name of some potential Page record
+  # @return [Boolean] whether or not a given Page record exists with a given 
+  #   name
+  def has_page?(name)
+    pages.exists?(:name => name)
+  end
+
   def has_redirect_for?(name)
      WikiReference.page_that_redirects_for(self, name) 
   end
@@ -52,11 +79,11 @@ class Web < ActiveRecord::Base
   end
 
   def has_file?(file_name)
-    WikiFile.find_by_file_name(file_name) != nil
+    wiki_files.exists?(:file_name => file_name)
   end
-  
-  def file_list(sort_order = 'file_name')
-    WikiFile.all(:order => sort_order, :conditions => ['web_id = ?', id])
+
+  def file_list(sort_order="file_name")
+    wiki_files.all(:order => sort_order)
   end
 
   def pages_that_link_to(page_name)
@@ -67,68 +94,83 @@ class Web < ActiveRecord::Base
     WikiReference.pages_that_link_to_file(self, file_name)
   end
 
+  # @param [String] file_name the name of some WikiFile of interest
+  # @return [String, nil] the description of some WikiFile of interest, nil if 
+  #   the WikiFile could not be found
   def description(file_name)
-    file = WikiFile.find_by_file_name(file_name)
-    file.description if file
+    wiki_files.find_by_file_name(file_name).try(:description)
   end
 
+  # @return [Symbol] the type of markup used by this Web
   def markup
-    read_attribute('markup').to_sym
+    self[:markup].to_sym
   end
 
+  # @return [Hash] a Hash wherein the key is some author's name, and the 
+  #   values are an array of page names for that author.
   def page_names_by_author
-    connection.select_all(
-        'SELECT DISTINCT r.author AS author, p.name AS page_name ' +
-        'FROM revisions r ' +
-        'JOIN pages p ON r.page_id = p.id ' +
-        "WHERE p.web_id = #{self.id} " +
-        'ORDER by p.name'
-    ).inject({}) { |result, row|
-        author, page_name = row['author'], row['page_name']
-        result[author] = [] unless result.has_key?(author)
-        result[author] << page_name
-        result
-    }
-  end
+    data = revisions.all(
+      :select => "DISTINCT revisions.author AS author, pages.name AS page_name",
+      :order  => "pages.name"
+    )
 
-  def remove_pages(pages_to_be_removed)
-    pages_to_be_removed.each { |p| p.destroy }
-  end
-  
-  def revised_at
-    select.most_recent_revision
-  end
-    
-  def select(&condition)
-    PageSet.new(self, pages, condition)
-  end
-  
-  def select_all
-    PageSet.new(self, pages, nil)
-  end
-  
-  def to_param
-    address
-  end
-  
-  def create_files_directory
-    return unless allow_uploads == 1
-    dummy_file = self.wiki_files.build(:file_name => '0', :description => '0', :content => '0')
-    File.umask(0002)
-    dir = File.dirname(dummy_file.content_path)
-    begin
-      require 'fileutils'
-      FileUtils.mkdir_p dir
-      dummy_file.save
-      dummy_file.destroy
-    rescue => e
-      logger.error("Failed create files directory for #{self.address}: #{e}")
-      raise "Instiki could not create directory to store uploaded files. " +
-            "Please make sure that Instiki is allowed to create directory " +
-            "#{File.expand_path(dir)} and add files to it."
+    data.inject({}) do |result, revision|
+      result[revision.author] ||= []
+      result[revision.author] <<  revision.page_name
+      result
     end
   end
 
+  # OPTIMIZE Use the +delete_all+ with conditions for extra efficiency
+  def remove_pages(pages_to_be_removed)
+    pages_to_be_removed.each { |p| p.destroy }
+  end
+
+  def revised_at
+    select.most_recent_revision
+  end
+
+  def select(&condition)
+    PageSet.new(self, pages, condition)
+  end
+
+  def select_all
+    PageSet.new(self, pages, nil)
+  end
+
+  # @return [String] uses the +address+ attribute for this record's parameter name
+  def to_param
+    address
+  end
+
+  # Called by an +after_save+ hook. Creates the directory that houses this 
+  # Web's associated files.
+  #
+  # TODO Move this into the WikiFile model
+  def create_files_directory
+    return unless allow_uploads == 1
+
+    dummy_file = wiki_files.build(
+      :file_name   => "0",
+      :description => "0",
+      :content     => "0"
+    )
+
+    File.umask(0002)
+
+    begin
+      dummy_file.content_path.parent.mkpath
+      dummy_file.save
+      dummy_file.destroy
+    rescue => e
+      logger.error "Failed create files directory for #{address}: #{e}"
+      raise "Instiki could not create directory to store uploaded files. " +
+            "Please make sure that Instiki is allowed to create directory " +
+            "#{dummy_file.content_path.expand_path} and add files to it."
+    end
+  end
+
+  # @return [Pathname] the path to the files for this record
   def files_path
     path = Rails.root.join("webs")
     if default_web?
@@ -138,6 +180,7 @@ class Web < ActiveRecord::Base
     end
   end
 
+  # @return [Pathname] the path to PNGs for this record
   def blahtex_pngs_path
     files_path.join("pngs")
   end
@@ -148,31 +191,31 @@ class Web < ActiveRecord::Base
     def wiki_words
       pages.inject([]) { |wiki_words, page| wiki_words << page.wiki_words }.flatten.uniq
     end
-    
+
     # Returns an array of all the page names on this web
     def page_names
       pages.map { |p| p.name }
     end
-    
+
   protected
-    before_save :sanitize_markup
-    after_save :create_files_directory
-    before_validation :validate_address
-    validates_uniqueness_of :address
-    validates_length_of :color, :in => 3..6
-  
+
     def sanitize_markup
       self.markup = markup.to_s
     end
-    
+
     def validate_address
+      if ['create_system', 'create_web', 'delete_web', 'delete_files', 'web_list', ''].include?(address)
+         self.errors.add(:address, 'is not a valid address')
+        raise Instiki::ValidationError.new("\"#{address.purify.escapeHTML}\" #{errors.on(:address)}")     
+      end
       unless address == CGI.escape(address)
         self.errors.add(:address, 'should contain only valid URI characters')
         raise Instiki::ValidationError.new("#{self.class.human_attribute_name('address')} #{errors.on(:address)}")
       end
     end
-    
+
+    # @return [Boolean] whether or not this record is considered the default Web
     def default_web?
-      defined? DEFAULT_WEB and self.address == DEFAULT_WEB
+      defined?(DEFAULT_WEB) && address == DEFAULT_WEB
     end
 end

app/models/wiki.rb

@@ -1,7 +1,7 @@
 class Wiki 
 
   cattr_accessor :storage_path, :logger
-  self.storage_path = "#{RAILS_ROOT}/storage/"
+  self.storage_path = Rails.root.join('storage')
 
   def authenticate(password)
     password == (system.password || 'instiki')
@@ -27,14 +27,28 @@ class Wiki
     if not (web = Web.find_by_address(old_address))
       raise Instiki::ValidationError.new("Web with address '#{old_address}' does not exist")
     end
-
+    old_files_path = web.files_path
+    
     web.update_attributes(:address => new_address, :name => name, :markup => markup, :color => color, 
       :additional_style => additional_style, :safe_mode => safe_mode, :password => password, :published => published,
       :brackets_only => brackets_only, :count_pages => count_pages, :allow_uploads => allow_uploads, :max_upload_size => max_upload_size)
     @webs = nil
     raise Instiki::ValidationError.new("There is already a web with address '#{new_address}'") unless web.errors.on(:address).nil?
     web
+    move_files(old_files_path, web.files_path)
   end
+  
+  def move_files(old_path, new_path)
+    return if new_path == old_path
+    default_path = Rails.root.join("webs", "files")
+    FileUtils.rmdir(new_path) if File.exist?(new_path)
+    if [old_path, new_path].include? default_path
+      File.rename(old_path, new_path)
+      FileUtils.rmdir(old_path.parent) unless old_path == default_path
+    else
+      File.rename(old_path.parent, new_path.parent)
+    end
+   end
 
   def read_page(web_address, page_name)
     ApplicationController.logger.debug "Reading page '#{page_name}' from web '#{web_address}'"

app/models/wiki_file.rb

@@ -9,13 +9,12 @@ class WikiFile < ActiveRecord::Base
   validates_length_of :description, :maximum=>255
 
   def self.find_by_file_name(file_name)
-    find(:first, :conditions => ['file_name = ?', file_name])
+    first(:conditions => ['file_name = ?', file_name])
   end
 
-  SANE_FILE_NAME = /^[a-zA-Z0-9\-_\. ]*$/
   def validate
     if file_name 
-      if file_name !~ SANE_FILE_NAME
+      if ! WikiFile.is_valid?(file_name)
         errors.add("file_name", "is invalid. Only latin characters, digits, dots, underscores, " +
            "dashes and spaces are accepted")
       elsif file_name == '.' or file_name == '..'
@@ -58,7 +57,10 @@ class WikiFile < ActiveRecord::Base
     require 'fileutils'
     FileUtils.rm_f(content_path) if File.exists?(content_path)
   end
-  
-  
+
+  SANE_FILE_NAME = /^[a-zA-Z0-9\-_\. ]*$/  
+  def self.is_valid?(name)
+    name =~ SANE_FILE_NAME
+  end
   
 end

app/models/wiki_reference.rb

@@ -12,6 +12,10 @@ class WikiReference < ActiveRecord::Base
   belongs_to :page
   validates_inclusion_of :link_type, :in => [LINKED_PAGE, WANTED_PAGE, REDIRECTED_PAGE, INCLUDED_PAGE, CATEGORY, AUTHOR, FILE, WANTED_FILE]
 
+  def referenced_name
+    read_attribute(:referenced_name).as_utf8
+  end
+
   def self.link_type(web, page_name)
     if web.has_page?(page_name) || self.page_that_redirects_for(web, page_name)
       LINKED_PAGE
@@ -42,7 +46,7 @@ class WikiReference < ActiveRecord::Base
     query = 'SELECT name FROM pages JOIN wiki_references ' +
       'ON pages.id = wiki_references.page_id ' +
       'WHERE wiki_references.referenced_name = ? ' +
-      "AND wiki_references.link_type in ('#{FILE}') " +
+      "AND wiki_references.link_type in ('#{FILE}','#{WANTED_FILE}') " +
       "AND pages.web_id = '#{web.id}'"
     names = connection.select_all(sanitize_sql([query, file_name])).map { |row| row['name'] }
   end
@@ -74,9 +78,9 @@ class WikiReference < ActiveRecord::Base
       "AND wiki_references.link_type = '#{REDIRECTED_PAGE}' " +
       "AND pages.web_id = '#{web.id}'"
     row = connection.select_one(sanitize_sql([query, page_name]))
-    row['name'] if row
+    row['name'].as_utf8 if row
   end
-  
+
   def self.pages_in_category(web, category)
     query = 
       "SELECT name FROM pages JOIN wiki_references " +
@@ -84,7 +88,7 @@ class WikiReference < ActiveRecord::Base
       "WHERE wiki_references.referenced_name = ? " +
       "AND wiki_references.link_type = '#{CATEGORY}' " +
       "AND pages.web_id = '#{web.id}'"
-    names = connection.select_all(sanitize_sql([query, category])).map { |row| row['name'] }
+    names = connection.select_all(sanitize_sql([query, category])).map { |row| row['name'].as_utf8 }
   end
   
   def self.list_categories(web)
@@ -93,7 +97,7 @@ class WikiReference < ActiveRecord::Base
       "ON wiki_references.page_id = pages.id " +
       "WHERE wiki_references.link_type = '#{CATEGORY}' " +
       "AND pages.web_id = '#{web.id}'"
-    connection.select_all(query).map { |row| row['referenced_name'] }
+    connection.select_all(query).map { |row| row['referenced_name'].as_utf8 }
   end
 
   def wiki_word?
@@ -128,4 +132,8 @@ class WikiReference < ActiveRecord::Base
     link_type == WANTED_FILE
   end
 
+  def category?
+    link_type == CATEGORY
+  end
+
 end

app/views/admin/edit_web.rhtml

@@ -18,7 +18,7 @@
                  onchange="proposeAddress();" />   
     <label for="address">Address:</label> <input type="text" class="disableAutoComplete" id="address" name="address" value="<%= @web.address %>" 
                     onchange="cleanAddress();" />
-    <small><em>(Letters and digits only)</em></small>
+    <em>(Letters and digits only)</em>
   </div>
 
   <h2 style="margin-bottom: 3px">Specialize</h2>
@@ -46,18 +46,18 @@
         }, @web.color) %>
     </select>
     <p>
-      <input type="checkbox" class="disableAutoComplete" id="safe_mode" name="safe_mode" <%= 'checked="checked"' if @web.safe_mode? %> /> 
+      <input type="checkbox" class="disableAutoComplete" id="safe_mode" name="safe_mode" <%= raw 'checked="checked"' if @web.safe_mode? %> /> 
       <label for="safe_mode">Safe mode
       <em>- strip HTML tags and stylesheet options from the content of all pages</em></label>
       <br/>
-      <input type="checkbox" class="disableAutoComplete" id="brackets_only" name="brackets_only" <%= 'checked="checked"' if @web.brackets_only? %> /> 
+      <input type="checkbox" class="disableAutoComplete" id="brackets_only" name="brackets_only" <%= raw 'checked="checked"' if @web.brackets_only? %> /> 
       <label for="brackets_only">Brackets only 
       <em>- require all wiki words to be as [[wiki word]], WikiWord links won't be created</em></label>
       <br/>
-      <input type="checkbox" class="disableAutoComplete" id="count_pages" name="count_pages" <%= 'checked="checked"' if @web.count_pages? %> /> 
+      <input type="checkbox" class="disableAutoComplete" id="count_pages" name="count_pages" <%= raw 'checked="checked"' if @web.count_pages? %> /> 
       <label for="count_pages">Count pages</label>
       <br/>
-      <input type="checkbox" class="disableAutoComplete" name="allow_uploads" <%= 'checked="checked"' if @web.allow_uploads? %> /> 
+      <input type="checkbox" class="disableAutoComplete" name="allow_uploads" <%= raw 'checked="checked"' if @web.allow_uploads? %> /> 
       Allow uploads of no more than 
       <input type="text" class="disableAutoComplete" name="max_upload_size" value="<%= @web.max_upload_size %>" 
              size="20" /> 
@@ -71,7 +71,8 @@
     Stylesheet tweaks &gt;&gt;</a>
     <em>
     - add or change styles used by this web; styles defined here take precedence over 
-    instiki.css. Hint: View HTML source of a page you want to style to find ID names on individual 
+    instiki.css.<br/>
+    Hint: View HTML source of a page you want to style to find ID names on individual 
     tags.</em>
     <br/>
     <textarea id="additionalStyle" class="disableAutoComplete" cols="50" rows="20"
@@ -96,10 +97,10 @@
   <div class="help">
     You can turn on a read-only version of this web that's accessible even when the regular web 
     is password protected.
-    The published version is accessible through URLs like /wiki/published/HomePage.
+    The published version is accessible through URLs like /<%= @web.address %>/published/HomePage.
   </div>
   <div class="inputBox">
-    <input type="checkbox" id="published" name="published" class="disableAutoComplete" <%= 'checked="checked"' if @web.published? %> /> 
+    <input type="checkbox" id="published" name="published" class="disableAutoComplete" <%= raw 'checked="checked"' if @web.published? %> /> 
     <label for="published">Publish this web</label>
   </div>
 

app/views/file/delete.html.erb

@@ -1,5 +1,5 @@
 <%-
-  @title = "Delete #{@file_name}"
+  @title = "Delete #{@file_name}".html_safe
   @hide_navigation = true
 -%>
 

app/views/file/import.rhtml

@@ -15,7 +15,7 @@
     <input type="text" name="author" id="authorName" value="<%= @author %>" 
         onclick="this.value == 'AnonymousCoward' ? this.value = '' : true" />
 	<%- if @page -%>
-      | <%= link_to 'Cancel', :web => @web.address, :action => 'file'%> <small>(unlocks page)</small>
+      | <%= link_to 'Cancel', :web => @web.address, :action => 'file'%> <em>(unlocks page)</em>
     <%- end -%>
     
   </p>

app/views/layouts/default.rhtml

@@ -16,19 +16,42 @@
   
   <%= javascript_include_tag 'page_helper' %>
  
+  <%= stylesheet_link_tag 'instiki', :media => 'all'  unless @inline_style %>
+  <%= stylesheet_link_tag 'syntax', :media => 'all'  unless @inline_style %>
   <style type="text/css">
-    h1#pageName, div.info, .newWikiWord a, a.existingWikiWord, .newWikiWord a:hover, #TextileHelp h3 { 
+    h1#pageName, div.info, .newWikiWord a, a.existingWikiWord, .newWikiWord a:hover, [actiontype="toggle"]:hover, #TextileHelp h3 { 
       color: #<%= @web ? @web.color : "393" %>; 
     }
-    <%= File.read(RAILS_ROOT + '/public/stylesheets/instiki.css') if @inline_style %>
+    a:visited.existingWikiWord {
+      color: #<%= darken(@web ? @web.color : "393") %>;
+    }
+    <%= Rails.root.join('public', 'stylesheets', 'instiki.css').read if @inline_style %>
+    <%= Rails.root.join('public', 'stylesheets', 'syntax.css').read if @inline_style %>
   </style>
-  <%= stylesheet_link_tag 'instiki' unless @inline_style %>
-  <%= "<style type='text/css'>#{@style_additions}</style>" if @style_additions %>
+  <%= "<style type='text/css'>#{@style_additions}</style>".html_safe if @style_additions %>
   <style type="text/css"><!--/*--><![CDATA[/*><!--*/    
-    <%= @web ? @web.additional_style : '' %>
+    <%= @web && @web.additional_style ? @web.additional_style.html_safe : '' %>
   /*]]>*/--></style>
   <%= javascript_include_tag :defaults %>
+  <%= csrf_meta_tag %>
   <%- if @web -%>
+   <%- if @web.markup == :markdownMML -%>
+  <script type="text/x-mathjax-config">
+    MathJax.Hub.Config({
+      MathML: { useMathMLspacing: true },
+      "HTML-CSS": { scale: 90 }
+    });
+    if (window._onload_fired_) MathJax.Hub.Startup.onload();
+  </script>
+  <script type="text/javascript">
+    if (!(Prototype.Browser.Gecko || navigator.userAgent.match(/MathPlayer/))) {
+      var s = document.createElement('script');
+      s.src = "<%= compute_public_path('MathJax.js', 'MathJax').split('?')[0] %>?config=MML_HTMLorMML";
+      document.querySelector('head').appendChild(s);
+      window.addEventListener("load", function(){window._onload_fired_ = true} , false);        
+    };
+  </script>
+   <%- end -%>
   	<%= auto_discovery_link_tag(:atom, {:controller => 'wiki', :web => @web.address, :action => 'atom_with_headlines'},
   	      :title => 'Atom with headlines') %>
   	<%= auto_discovery_link_tag(:atom, {:controller => 'wiki', :web => @web.address, :action => 'atom_with_content'},
@@ -41,7 +64,7 @@
 <div id="Container">
 <div id="Content">
   <h1 id="pageName">
-    <%= render(:file => 'svg_logo') if @controller.xhtml_enabled? %>
+    <%= render(:file => 'svg_logo') if xhtml_enabled? %>
     <%- if @page and (@page.name == 'HomePage') and %w( show published print ).include?(@action_name) -%>
       <%= h(@web.name) + (@show_diff ? ' (changes)' : '') %>
     <%- elsif @web -%>

app/views/layouts/error.html.erb

@@ -12,10 +12,10 @@
     h1#pageName, .newWikiWord a, a.existingWikiWord, .newWikiWord a:hover, #TextileHelp h3 { 
       color: #<%= @web ? @web.color : "393" %>; 
     }
-    <%= File.read(RAILS_ROOT + '/public/stylesheets/instiki.css') if @inline_style %>
+    <%= Rails.root.join('public', 'stylesheets', 'instiki.css').read if @inline_style %>
   </style>
 
-  <%= stylesheet_link_tag 'instiki' unless @inline_style %>
+  <%= stylesheet_link_tag 'instiki', :media => 'all' unless @inline_style %>
 
   <style type="text/css"><!--/*--><![CDATA[/*><!--*/
     <%= @style_additions %>
@@ -31,7 +31,11 @@
   </h1>
 
 <div id="Error-Content">
-<%= h @content_for_layout %>
+<%= if :raw
+      @content_for_layout
+    else
+      h @content_for_layout
+    end %>
 
 </div> <!-- Error-Content -->
 

app/views/layouts/tex.rhtml

@@ -1 +1,265 @@
+\documentclass[12pt,titlepage]{article}
+
+\usepackage{amsmath}
+\usepackage{amsfonts}
+\usepackage{amssymb}
+\usepackage{amsthm}
+\usepackage{mathtools}
+<%- if @tex_content =~  /\\mathbb\{[^}]*[a-z0-9]+[^}]*\}/ -%>
+\usepackage{mathbbol}
+<% end -%>
+\usepackage{graphicx}
+\usepackage{color}
+\usepackage{ucs}
+\usepackage[utf8x]{inputenc}
+\usepackage{xparse}
+\usepackage{hyperref}
+
+%----Macros----------
+%
+% Unresolved issues:
+%
+%  \righttoleftarrow
+%  \lefttorightarrow
+%
+%  \color{} with HTML colorspec
+%  \bgcolor
+%  \array with options (without options, it's equivalent to the matrix environment)
+
+% Of the standard HTML named colors, white, black, red, green, blue and yellow
+% are predefined in the color package. Here are the rest.
+\definecolor{aqua}{rgb}{0, 1.0, 1.0}
+\definecolor{fuschia}{rgb}{1.0, 0, 1.0}
+\definecolor{gray}{rgb}{0.502, 0.502, 0.502}
+\definecolor{lime}{rgb}{0, 1.0, 0}
+\definecolor{maroon}{rgb}{0.502, 0, 0}
+\definecolor{navy}{rgb}{0, 0, 0.502}
+\definecolor{olive}{rgb}{0.502, 0.502, 0}
+\definecolor{purple}{rgb}{0.502, 0, 0.502}
+\definecolor{silver}{rgb}{0.753, 0.753, 0.753}
+\definecolor{teal}{rgb}{0, 0.502, 0.502}
+
+% Because of conflicts, \space and \mathop are converted to
+% \itexspace and \operatorname during preprocessing.
+
+% itex: \space{ht}{dp}{wd}
+%
+% Height and baseline depth measurements are in units of tenths of an ex while
+% the width is measured in tenths of an em.
+\makeatletter
+\newdimen\itex@wd%
+\newdimen\itex@dp%
+\newdimen\itex@thd%
+\def\itexspace#1#2#3{\itex@wd=#3em%
+\itex@wd=0.1\itex@wd%
+\itex@dp=#2ex%
+\itex@dp=0.1\itex@dp%
+\itex@thd=#1ex%
+\itex@thd=0.1\itex@thd%
+\advance\itex@thd\the\itex@dp%
+\makebox[\the\itex@wd]{\rule[-\the\itex@dp]{0cm}{\the\itex@thd}}}
+\makeatother
+
+% \tensor and \multiscript
+\makeatletter
+\newif\if@sup
+\newtoks\@sups
+\def\append@sup#1{\edef\act{\noexpand\@sups={\the\@sups #1}}\act}%
+\def\reset@sup{\@supfalse\@sups={}}%
+\def\mk@scripts#1#2{\if #2/ \if@sup ^{\the\@sups}\fi \else%
+  \ifx #1_ \if@sup ^{\the\@sups}\reset@sup \fi {}_{#2}%
+  \else \append@sup#2 \@suptrue \fi%
+  \expandafter\mk@scripts\fi}
+\def\tensor#1#2{\reset@sup#1\mk@scripts#2_/}
+\def\multiscripts#1#2#3{\reset@sup{}\mk@scripts#1_/#2%
+  \reset@sup\mk@scripts#3_/}
+\makeatother
+
+% \slash
+\makeatletter
+\newbox\slashbox \setbox\slashbox=\hbox{$/$}
+\def\itex@pslash#1{\setbox\@tempboxa=\hbox{$#1$}
+  \@tempdima=0.5\wd\slashbox \advance\@tempdima 0.5\wd\@tempboxa
+  \copy\slashbox \kern-\@tempdima \box\@tempboxa}
+\def\slash{\protect\itex@pslash}
+\makeatother
+
+% math-mode versions of \rlap, etc
+% from Alexander Perlis, "A complement to \smash, \llap, and lap"
+%   http://math.arizona.edu/~aprl/publications/mathclap/
+\def\clap#1{\hbox to 0pt{\hss#1\hss}}
+\def\mathllap{\mathpalette\mathllapinternal}
+\def\mathrlap{\mathpalette\mathrlapinternal}
+\def\mathclap{\mathpalette\mathclapinternal}
+\def\mathllapinternal#1#2{\llap{$\mathsurround=0pt#1{#2}$}}
+\def\mathrlapinternal#1#2{\rlap{$\mathsurround=0pt#1{#2}$}}
+\def\mathclapinternal#1#2{\clap{$\mathsurround=0pt#1{#2}$}}
+
+% Renames \sqrt as \oldsqrt and redefine root to result in \sqrt[#1]{#2}
+\let\oldroot\root
+\def\root#1#2{\oldroot #1 \of{#2}}
+\renewcommand{\sqrt}[2][]{\oldroot #1 \of{#2}}
+
+% Manually declare the txfonts symbolsC font
+\DeclareSymbolFont{symbolsC}{U}{txsyc}{m}{n}
+\SetSymbolFont{symbolsC}{bold}{U}{txsyc}{bx}{n}
+\DeclareFontSubstitution{U}{txsyc}{m}{n}
+
+% Manually declare the stmaryrd font
+\DeclareSymbolFont{stmry}{U}{stmry}{m}{n}
+\SetSymbolFont{stmry}{bold}{U}{stmry}{b}{n}
+
+% Declare specific arrows from txfonts without loading the full package
+\makeatletter
+\def\re@DeclareMathSymbol#1#2#3#4{%
+    \let#1=\undefined
+    \DeclareMathSymbol{#1}{#2}{#3}{#4}}
+\re@DeclareMathSymbol{\neArrow}{\mathrel}{symbolsC}{116}
+\re@DeclareMathSymbol{\neArr}{\mathrel}{symbolsC}{116}
+\re@DeclareMathSymbol{\seArrow}{\mathrel}{symbolsC}{117}
+\re@DeclareMathSymbol{\seArr}{\mathrel}{symbolsC}{117}
+\re@DeclareMathSymbol{\nwArrow}{\mathrel}{symbolsC}{118}
+\re@DeclareMathSymbol{\nwArr}{\mathrel}{symbolsC}{118}
+\re@DeclareMathSymbol{\swArrow}{\mathrel}{symbolsC}{119}
+\re@DeclareMathSymbol{\swArr}{\mathrel}{symbolsC}{119}
+\re@DeclareMathSymbol{\nequiv}{\mathrel}{symbolsC}{46}
+\re@DeclareMathSymbol{\Perp}{\mathrel}{symbolsC}{121}
+\re@DeclareMathSymbol{\Vbar}{\mathrel}{symbolsC}{121}
+\re@DeclareMathSymbol{\sslash}{\mathrel}{stmry}{12}
+\re@DeclareMathSymbol{\invamp}{\mathrel}{symbolsC}{77}
+\re@DeclareMathSymbol{\parr}{\mathrel}{symbolsC}{77}
+\makeatother
+
+% Widecheck
+\makeatletter
+\DeclareRobustCommand\widecheck[1]{{\mathpalette\@widecheck{#1}}}
+\def\@widecheck#1#2{%
+    \setbox\z@\hbox{\m@th$#1#2$}%
+    \setbox\tw@\hbox{\m@th$#1%
+       \widehat{%
+          \vrule\@width\z@\@height\ht\z@
+          \vrule\@height\z@\@width\wd\z@}$}%
+    \dp\tw@-\ht\z@
+    \@tempdima\ht\z@ \advance\@tempdima2\ht\tw@ \divide\@tempdima\thr@@
+    \setbox\tw@\hbox{%
+       \raise\@tempdima\hbox{\scalebox{1}[-1]{\lower\@tempdima\box
+\tw@}}}%
+    {\ooalign{\box\tw@ \cr \box\z@}}}
+\makeatother
+
+% \mathraisebox{voffset}[height][depth]{something}
+\makeatletter
+\NewDocumentCommand\mathraisebox{moom}{%
+\IfNoValueTF{#2}{\def\@temp##1##2{\raisebox{#1}{$\m@th##1##2$}}}{%
+\IfNoValueTF{#3}{\def\@temp##1##2{\raisebox{#1}[#2]{$\m@th##1##2$}}%
+}{\def\@temp##1##2{\raisebox{#1}[#2][#3]{$\m@th##1##2$}}}}%
+\mathpalette\@temp{#4}}
+\makeatletter
+
+% udots (taken from yhmath)
+\makeatletter
+\def\udots{\mathinner{\mkern2mu\raise\p@\hbox{.}
+\mkern2mu\raise4\p@\hbox{.}\mkern1mu
+\raise7\p@\vbox{\kern7\p@\hbox{.}}\mkern1mu}}
+\makeatother
+
+%% Fix array
+\newcommand{\itexarray}[1]{\begin{matrix}#1\end{matrix}}
+%% \itexnum is a noop
+\newcommand{\itexnum}[1]{#1}
+
+%% Renaming existing commands
+\newcommand{\underoverset}[3]{\underset{#1}{\overset{#2}{#3}}}
+\newcommand{\widevec}{\overrightarrow}
+\newcommand{\darr}{\downarrow}
+\newcommand{\nearr}{\nearrow}
+\newcommand{\nwarr}{\nwarrow}
+\newcommand{\searr}{\searrow}
+\newcommand{\swarr}{\swarrow}
+\newcommand{\curvearrowbotright}{\curvearrowright}
+\newcommand{\uparr}{\uparrow}
+\newcommand{\downuparrow}{\updownarrow}
+\newcommand{\duparr}{\updownarrow}
+\newcommand{\updarr}{\updownarrow}
+\newcommand{\gt}{>}
+\newcommand{\lt}{<}
+\newcommand{\map}{\mapsto}
+\newcommand{\embedsin}{\hookrightarrow}
+\newcommand{\Alpha}{A}
+\newcommand{\Beta}{B}
+\newcommand{\Zeta}{Z}
+\newcommand{\Eta}{H}
+\newcommand{\Iota}{I}
+\newcommand{\Kappa}{K}
+\newcommand{\Mu}{M}
+\newcommand{\Nu}{N}
+\newcommand{\Rho}{P}
+\newcommand{\Tau}{T}
+\newcommand{\Upsi}{\Upsilon}
+\newcommand{\omicron}{o}
+\newcommand{\lang}{\langle}
+\newcommand{\rang}{\rangle}
+\newcommand{\Union}{\bigcup}
+\newcommand{\Intersection}{\bigcap}
+\newcommand{\Oplus}{\bigoplus}
+\newcommand{\Otimes}{\bigotimes}
+\newcommand{\Wedge}{\bigwedge}
+\newcommand{\Vee}{\bigvee}
+\newcommand{\coproduct}{\coprod}
+\newcommand{\product}{\prod}
+\newcommand{\closure}{\overline}
+\newcommand{\integral}{\int}
+\newcommand{\doubleintegral}{\iint}
+\newcommand{\tripleintegral}{\iiint}
+\newcommand{\quadrupleintegral}{\iiiint}
+\newcommand{\conint}{\oint}
+\newcommand{\contourintegral}{\oint}
+\newcommand{\infinity}{\infty}
+\newcommand{\bottom}{\bot}
+\newcommand{\minusb}{\boxminus}
+\newcommand{\plusb}{\boxplus}
+\newcommand{\timesb}{\boxtimes}
+\newcommand{\intersection}{\cap}
+\newcommand{\union}{\cup}
+\newcommand{\Del}{\nabla}
+\newcommand{\odash}{\circleddash}
+\newcommand{\negspace}{\!}
+\newcommand{\widebar}{\overline}
+\newcommand{\textsize}{\normalsize}
+\renewcommand{\scriptsize}{\scriptstyle}
+\newcommand{\scriptscriptsize}{\scriptscriptstyle}
+\newcommand{\mathfr}{\mathfrak}
+\newcommand{\statusline}[2]{#2}
+\newcommand{\tooltip}[2]{#2}
+\newcommand{\toggle}[2]{#2}
+
+% Theorem Environments
+\theoremstyle{plain}
+\newtheorem{theorem}{Theorem}
+\newtheorem{lemma}{Lemma}
+\newtheorem{prop}{Proposition}
+\newtheorem{cor}{Corollary}
+\newtheorem*{utheorem}{Theorem}
+\newtheorem*{ulemma}{Lemma}
+\newtheorem*{uprop}{Proposition}
+\newtheorem*{ucor}{Corollary}
+\theoremstyle{definition}
+\newtheorem{defn}{Definition}
+\newtheorem{example}{Example}
+\newtheorem*{udefn}{Definition}
+\newtheorem*{uexample}{Example}
+\theoremstyle{remark}
+\newtheorem{remark}{Remark}
+\newtheorem{note}{Note}
+\newtheorem*{uremark}{Remark}
+\newtheorem*{unote}{Note}
+
+%-------------------------------------------------------------------
+
+\begin{document}
+
+%-------------------------------------------------------------------
+
 <%= @content_for_layout %>
+
+\end{document}

app/views/markdownMML_help.rhtml

@@ -1,15 +1,3 @@
-<h3>Markdown+itex2MML formatting tips (<a target="_new" href="http://daringfireball.net/projects/markdown/syntax">basics</a>, <a target="_new" href="http://maruku.rubyforge.org/maruku.html#extra">extended syntax</a>, <a target="_new" href="http://maruku.rubyforge.org/proposal.html">metadata</a>, <a target="_new" href="http://golem.ph.utexas.edu/instiki/show/Theorems">theorems</a>)</h3>
+<h3>Markdown+itex2MML formatting tips (<a target="_new" href="http://daringfireball.net/projects/markdown/syntax">basics</a>, <a target="_new" href="http://michelf.com/projects/php-markdown/extra/">extended syntax</a>, <a target="_new" href="http://maruku.rubyforge.org/proposal.html">metadata</a>, <a target="_new" href="http://golem.ph.utexas.edu/instiki/show/Theorems">theorems</a>)</h3>
 <p>For a complete list of LaTeX commands supported here, see the <a href="http://golem.ph.utexas.edu/~distler/blog/itex2MMLcommands.html">itex2MML Commands Summary</a>.</p>
-<table cellspacing="0" cellpadding="0">
-  <tr><td>_your text_</td><td class="arrow">&#x2192;</td><td><em>your text</em></td></tr>
-  <tr><td>**your text**</td><td class="arrow">&#x2192;</td><td><strong>your text</strong></td></tr>
-  <tr><td>`my code`</td><td class="arrow">&#x2192;</td><td><code>my code</code></td></tr>
-  <tr><td>* Bulleted list<br />* Second item</td><td class="arrow">&#x2192;</td><td>&#8226; Bulleted list<br />&#8226; Second item</td></tr>
-  <tr><td>1. Numbered list<br />1. Second item</td><td class="arrow">&#x2192;</td><td>1. Numbered list<br />2. Second item</td></tr>
-  <tr><td>Definition list<br />: is useful</td><td class="arrow">&#x2192;</td><td><dl style="display:inline"><dt>Definition list</dt><dd>is useful</dd></dl></td></tr>
-  <tr><td>[link name](URL)</td><td class="arrow">&#x2192;</td><td><a href="URL">link name</a></td></tr>
-  <tr><td>![Alt text](URL)</td><td class="arrow">&#x2192;</td><td>Image</td></tr>
-  <tr><td>## Header ##<br />### Subheader ###<br />#### Etc. ####</td><td class="arrow">&#x2192;</td><td><b><span style="font-size:1.2em">Header</span><br /><span style="font-size:1.1em">Subheader</span><br /><span style="font-size:1em">Etc.</span></b></td></tr>
-  <tr><td>***</td><td class="arrow">&#x2192;</td><td>Horizontal ruler</td></tr>
-  <tr><td>&lt;http://url><br />&lt;email@add.com></td><td class="arrow">&#x2192;</td><td>Auto-linked</td></tr>
-</table>
+<%= render(:file => "markdown_table") -%>

app/views/markdownPNG_help.html.erb

@@ -1,16 +1,3 @@
-<h3>Markdown+blahtex/PNG formatting tips (<a target="_new" href="http://daringfireball.net/projects/markdown/syntax">basics</a>, <a target="_new" href="http://maruku.rubyforge.org/#extra">extended syntax</a>, <a target="_new" href="http://maruku.rubyforge.org/proposal.html">metadata</a>)</h3>
-<table cellspacing="0" cellpadding="0">
-  <tr><td>_your text_</td><td class="arrow">&#x2192;</td><td><em>your text</em></td></tr>
-  <tr><td>**your text**</td><td class="arrow">&#x2192;</td><td><strong>your text</strong></td></tr>
-  <tr><td>`my code`</td><td class="arrow">&#x2192;</td><td><code>my code</code></td></tr>
-  <tr><td>$LaTeX code$</td><td class="arrow">&#x2192;</td><td>Insert an inline <br /> math expression</td></tr>
-  <tr><td>\[LaTeX code\]</td><td class="arrow">&#x2192;</td><td>Insert a math <br /> expression on <br /> its own line</td></tr>
-  <tr><td>* Bulleted list<br />* Second item</td><td class="arrow">&#x2192;</td><td>&#8226; Bulleted list<br />&#8226; Second item</td></tr>
-  <tr><td>1. Numbered list<br />1. Second item</td><td class="arrow">&#x2192;</td><td>1. Numbered list<br />2. Second item</td></tr>
-  <tr><td>Definition list<br />: is useful</td><td class="arrow">&#x2192;</td><td><dl style="display:inline"><dt>Definition list</dt><dd>is useful</dd></dl></td></tr>
-  <tr><td>[link name](URL)</td><td class="arrow">&#x2192;</td><td><a href="URL">link name</a></td></tr>
-  <tr><td>![Alt text](URL)</td><td class="arrow">&#x2192;</td><td>Image</td></tr>
-  <tr><td>## Header ##<br />### Subheader ###<br />#### Etc. ####</td><td class="arrow">&#x2192;</td><td><b><span style="font-size:1.2em">Header</span><br /><span style="font-size:1.1em">Subheader</span><br /><span style="font-size:1em">Etc.</span></b></td></tr>
-  <tr><td>***</td><td class="arrow">&#x2192;</td><td>Horizontal ruler</td></tr>
-  <tr><td>&lt;http://url><br />&lt;email@add.com></td><td class="arrow">&#x2192;</td><td>Auto-linked</td></tr>
-</table>
+<h3>Markdown+blahtex/PNG formatting tips (<a target="_new" href="http://daringfireball.net/projects/markdown/syntax">basics</a>, <a target="_new" href="http://michelf.com/projects/php-markdown/extra/">extended syntax</a>, <a target="_new" href="http://maruku.rubyforge.org/proposal.html">metadata</a>)</h3>
+<p>For a list of the LaTeX commands supported here, see the <a href="http://www.blahtex.org/manual.html">BlahTeX manual</a>.</p>
+<%= render(:file => "markdown_table") -%>

app/views/markdown_help.rhtml

@@ -1,14 +1,2 @@
-<h3>Markdown formatting tips (<a target="_new" href="http://daringfireball.net/projects/markdown/syntax">basics</a>, <a target="_new" href="http://maruku.rubyforge.org/#extra">extended syntax</a>, <a target="_new" href="http://maruku.rubyforge.org/proposal.html">metadata</a>)</h3>
-<table cellspacing="0" cellpadding="0">
-  <tr><td>_your text_</td><td class="arrow">&#x2192;</td><td><em>your text</em></td></tr>
-  <tr><td>**your text**</td><td class="arrow">&#x2192;</td><td><strong>your text</strong></td></tr>
-  <tr><td>`my code`</td><td class="arrow">&#x2192;</td><td><code>my code</code></td></tr>
-  <tr><td>* Bulleted list<br />* Second item</td><td class="arrow">&#x2192;</td><td>&#8226; Bulleted list<br />&#8226; Second item</td></tr>
-  <tr><td>1. Numbered list<br />1. Second item</td><td class="arrow">&#x2192;</td><td>1. Numbered list<br />2. Second item</td></tr>
-  <tr><td>Definition list<br />: is useful</td><td class="arrow">&#x2192;</td><td><dl style="display:inline"><dt>Definition list</dt><dd>is useful</dd></dl></td></tr>
-  <tr><td>[link name](URL)</td><td class="arrow">&#x2192;</td><td><a href="URL">link name</a></td></tr>
-  <tr><td>![Alt text](URL)</td><td class="arrow">&#x2192;</td><td>Image</td></tr>
-  <tr><td>## Header ##<br />### Subheader ###<br />#### Etc. ####</td><td class="arrow">&#x2192;</td><td><b><span style="font-size:1.2em">Header</span><br /><span style="font-size:1.1em">Subheader</span><br /><span style="font-size:1em">Etc.</span></b></td></tr>
-  <tr><td>***</td><td class="arrow">&#x2192;</td><td>Horizontal ruler</td></tr>
-  <tr><td>&lt;http://url><br />&lt;email@add.com></td><td class="arrow">&#x2192;</td><td>Auto-linked</td></tr>
-</table>
+<h3>Markdown formatting tips (<a target="_new" href="http://daringfireball.net/projects/markdown/syntax">basics</a>, <a target="_new" href="http://michelf.com/projects/php-markdown/extra/">extended syntax</a>, <a target="_new" href="http://maruku.rubyforge.org/proposal.html">metadata</a>)</h3>
+<%= render(:file => "markdown_table") -%>

app/views/markdown_table.html.erb

@@ -0,0 +1,17 @@
+<table cellspacing="0" cellpadding="0">
+  <tr><td>_your text_</td><td class="arrow">&#x2192;</td><td><em>your text</em></td></tr>
+  <tr><td>**your text**</td><td class="arrow">&#x2192;</td><td><strong>your text</strong></td></tr>
+  <tr><td>`my code`</td><td class="arrow">&#x2192;</td><td><code>my code</code></td></tr>
+  <tr><td>* Bulleted list<br />* Second item</td><td class="arrow">&#x2192;</td><td>&#8226; Bulleted list<br />&#8226; Second item</td></tr>
+  <tr><td>1. Numbered list<br />1. Second item</td><td class="arrow">&#x2192;</td><td>1. Numbered list<br />2. Second item</td></tr>
+  <tr><td>Definition list<br />: is useful</td><td class="arrow">&#x2192;</td><td><dl style="display:inline"><dt>Definition list</dt><dd>is useful</dd></dl></td></tr>
+  <tr><td>[link name](URL)</td><td class="arrow">&#x2192;</td><td><a href="URL">link name</a></td></tr>
+  <tr><td>![Alt text](URL)</td><td class="arrow">&#x2192;</td><td>Image</td></tr>
+  <tr><td>## Header ##<br />### Subheader ###<br />#### Subsubhead####<br/>##### Etc. #####</td><td class="arrow">&#x2192;</td>
+      <td><b><span style="font-size:1.73em">Header</span>
+          <br/><span style="font-size:1.44em">Subheader</span><br/><span style="font-size:1.2em">Subsubhead</span>
+          <br/><span style="font-size:1em">Etc.</span></b></td></tr>
+  <tr><td>***</td><td class="arrow">&#x2192;</td><td>Horizontal rule</td></tr>
+  <tr><td>Some text[^fine] here.<br/>[^fine]: The fine print.</td><td class="arrow">&#x2192;</td><td>A footnote</td></tr>
+  <tr><td>*[LA]: Los Angeles<br/>*[ppm]: parts per million</td><td class="arrow">&#x2192;</td><td>Abbreviations</td></tr>
+</table>

app/views/navigation.rhtml

@@ -9,7 +9,8 @@ end
 %>
 
 <div class="navigation">
-  <% if params['action'] != 'published' then %>
+    <span class="skipNav"><a href='#navEnd'>Skip the Navigation Links</a> | </span>
+<%- if params['action'] != 'published' then -%>
     <%= list_item 'Home Page', {:action => 'show', :id => 'HomePage'}, 'Home, Sweet Home', 'H' %> | 
     <%= list_item 'All Pages', {:action => 'list'}, 'Alphabetically sorted list of pages', 'A' %> | 
     <%= list_item 'Recently Revised', {:action =>'recently_revised'}, 'Pages sorted by when they were last changed', 'U' %> |
@@ -22,8 +23,9 @@ end
              onfocus="this.value == 'Search' ? this.value = '' : true"
              onblur="this.value == '' ? this.value = 'Search' : true" /></fieldset>
     <% end %>
-  <% else %>
-  	<%= list_item 'Home Page', {:action => 'published', :id => 'HomePage'}, 'Home, Sweet Home', 'H' %> | 
+<%- else -%>
+    <%= list_item 'Home Page', {:action => 'published', :id => 'HomePage'}, 'Home, Sweet Home', 'H' %> | 
     <%= list_item 'Feeds', {:action => 'feeds'}, 'Subscribe to changes by Atom' %>
-  <% end%>
+<% end-%>
+  <span id='navEnd'></span>
 </div>

app/views/svg_logo.html.erb

@@ -1,4 +1,4 @@
-<span id="svg_logo"><svg version="1.1" width="100%" height="100%" viewBox='0 0 180 197' xmlns='http://www.w3.org/2000/svg'>
+<span id="svg_logo"><svg version="1.1" width="100%" height="100%" viewBox='0 -1 180 198' xmlns='http://www.w3.org/2000/svg'>
       <path id="svg_logo_path" fill="#<%= @web ? @web.color : "393" %>" stroke-width='0.5' stroke='#000' d='
         M170,60c4,11-1,20-12,25c-9,4-25,3-20,15c5,5,15,0,24,1c11,1,21,11,14,21c-10,15-35,6-48-1c-5-3-27-23-32-10c-1,13,15,10,22,16
         c11,4,24,14,34,20c12,10,7,25-9,23c-11-1-22-9-30-16c-5-5-13-18-21-9c-2,6,2,11,5,14c9,9,22,14,22,31c-2,8-12,8-18,4c-4-3-9-8-11-13

app/views/textile_help.rhtml

@@ -1,20 +1,19 @@
-<h3>Textile formatting tips (<a href="http://hobix.com/textile/quick.html" onclick="quickRedReference(); return false;">advanced</a>)</h3>
+<h3>Textile formatting tips (<a href="http://redcloth.org/hobix.com/textile/quick.html" onclick="quickRedReference(); return false;">advanced</a>)</h3>
 <table cellspacing="0" cellpadding="0">
   <tr><td>_your text_</td><td class="arrow">&rarr;</td><td><em>your text</em></td></tr>
   <tr><td>*your text*</td><td class="arrow">&rarr;</td><td><strong>your text</strong></td></tr>
-  <tr><td>%{color:red}hello%</td><td class="arrow">&rarr;</td><td><span style="color: red;">hello</span></td></tr>
+  <tr><td style="padding-right:0.75em">%{color:red}hello%</td><td class="arrow">&rarr;</td><td><span style="color: red;">hello</span></td></tr>
   <tr><td>* Bulleted list<br />* Second item</td><td class="arrow">&rarr;</td><td>&#8226; Bulleted list<br />&#8226; Second item</td></tr>
   <tr><td># Numbered list<br /># Second item</td><td class="arrow">&rarr;</td><td>1. Numbered list<br />2. Second item</td></tr>
   <tr><td>"linkname":URL</td><td class="arrow">&rarr;</td><td><a href="URL">linkname</a></td></tr>
   <tr><td>|a|table|row|<br />|b|table|row|</td><td class="arrow">&rarr;</td><td>Table</td></tr>
-  <tr><td>http://url<br />email@address.com</td><td class="arrow">&rarr;</td><td>Auto-linked</td></tr>
   <tr><td>!imageURL!</td><td class="arrow">&rarr;</td><td>Image</td></tr>
 </table>
     
 <script language="JavaScript">
   function quickRedReference() {
     window.open( 
-      "http://hobix.com/textile/quick.html",
+      "http://redcloth.org/hobix.com/textile/quick.html",
       "redRef",
       "height=600,width=550,channelmode=0,dependent=0," +
       "directories=0,fullscreen=0,location=0,menubar=0," +

app/views/wiki/_inbound_links.rhtml

@@ -1,13 +1,13 @@
 <%- unless @page.linked_from.empty? -%>
   <span class="linked">
     | Linked from: 
-    <%= @page.linked_from.collect { |referring_page| link_to_existing_page referring_page }.join(", ") %>
+    <%= @page.linked_from.collect { |referring_page| link_to_existing_page referring_page }.join(", ").html_safe %>
   </span>
 <%- end -%>
 
 <%- unless @page.included_from.empty? -%>
   <span class="linked">
     | Included from: 
-    <%= @page.included_from.collect { |referring_page| link_to_existing_page referring_page }.join(", ") %>
+    <%= @page.included_from.collect { |referring_page| link_to_existing_page referring_page }.join(", ").html_safe %>
   </span>
 <%- end -%>

app/views/wiki/authors.rhtml

@@ -5,7 +5,7 @@
     <li>
       <%= link_to_page author.purify %>
       co- or authored:
-      <%= @page_names_by_author[author].collect { |page_name| link_to_page(page_name) }.sort.join ', ' %>
+      <%= raw @page_names_by_author[author].collect { |page_name| link_to_page(page_name) }.sort.join ', ' %>
     </li>
   <%- end -%>
 </ul>

app/views/wiki/edit.rhtml

@@ -1,12 +1,12 @@
 <%- 
-  @title = "Editing #{@page.name.escapeHTML}"
+  @title = "Editing #{@page.name.escapeHTML}".html_safe
   @content_width = 720
   @hide_navigation = true
 -%>
 
 <div id="MarkupHelp">
-  <%= render(:file => "#{@web.markup}_help") %>
-  <%= render(:file => 'wiki_words_help') %>
+<%= render(:file => "#{@web.markup}_help") -%>
+<%= render(:file => 'wiki_words_help') unless @web.brackets_only? -%>
 </div>
 
 	<% form_tag({ :action => 'save', :web => @web.address, :id => @page.name },
@@ -14,7 +14,7 @@
 	               'accept-charset' => 'utf-8' }) do %>
 	   <div>	
 		<textarea name="content" id="content" rows="24" cols="60"><%= h(flash[:content] || 
-                         ((params['content'] && params['content'].is_utf8?) ?  params['content'] : @page.content).purify) %></textarea>
+                         (params['content'] ?  params['content'] : @page.content).purify) %></textarea>
 <% if @page_name != 'HomePage' -%>
 	   <p>
 	      <%= check_box_tag :alter_title, value = "1", checked=false,
@@ -55,7 +55,8 @@ function toggleVisibility() {
 }
 
 function addRedirect(){
-  if (document.getElementById('new_name').value != "<%= escape_javascript(@page.name) %>" ) {
+  var e = document.getElementById('new_name').value;
+  if ( e != "<%= escape_javascript(@page.name) %>" && e != '') {
     var content = document.getElementById('content');
     content.value = '[[!redirects <%= escape_javascript(@page.name) %>]]\n' + content.value
     }
@@ -68,5 +69,11 @@ function cleanAuthorName() {
 }
 
 document.forms["editForm"].elements["content"].focus();
+<%- if [:markdownMML, :markdownPNG, :markdown].include?(@web.markup) -%>
+setupSVGedit('<%= compute_public_path("editor/svg-editor.html", "svg-edit").split(/\?/)[0] %>');
+<%- unless @page.categories.include?('S5-slideshow') -%>
+addS5button('<%= @page.name.escapeHTML %>');
+<%- end -%>
+<%- end -%>
 //--><!]]>
 </script>

app/views/wiki/file_list.html.erb

@@ -20,7 +20,7 @@
       <a href="<%= url_for :web => @web.address, :action => 'files', 
             :id => file.file_name %>"><%= file.file_name%></a> (<%= file.created_at.asctime %>) <span class="linked"><%= "Linked to by: " unless
             @web.pages_that_link_to_file(file.file_name).empty? -%>
-            <%= @web.pages_that_link_to_file(file.file_name).collect { |referring_page|  link_to_page(referring_page) }.join(", ") %></span>
+            <%= @web.pages_that_link_to_file(file.file_name).collect { |referring_page|  link_to_page(referring_page) }.join(", ").html_safe %></span>
     </li>
   <%- end -%>
   </ul>

app/views/wiki/history.html.erb

@@ -1,4 +1,4 @@
-<%- @title = @page.plain_name + " (history)" -%>
+<%- @title = @page.plain_name + " (history)".html_safe -%>
 <%- @show_footer = true -%>
 
 <%- @revisions_by_day.keys.sort.reverse.each do |day| -%>
@@ -7,7 +7,7 @@
   <%- for rev in @revisions_by_day[day] -%>
     <li>
       <%= link_to_revision(rev.page, @revision_numbers[rev.id],
-          text= (rev.page.revisions.size == @revision_numbers[rev.id] ?
+          text= (rev.page.rev_ids.size == @revision_numbers[rev.id] ?
               "Current" :
               "Revision #{@revision_numbers[rev.id]}" )
               ) %>

app/views/wiki/list.rhtml

@@ -6,10 +6,34 @@
 <%- unless @pages_that_are_orphaned.empty? && @page_names_that_are_wanted.empty? -%>
   <h2>
     All Pages
-    <br/><span class="pageType">All pages in <%= @set_name %> listed alphabetically</span>
+    <br/><span class="pageType">All pages in <%= raw @set_name %> listed alphabetically</span>
   </h2>
 <%- end -%>
 
+<% if [:markdownMML, :markdownPNG, :markdown].include?(@web.markup) -%>
+<% form_tag({ :controller => 'wiki', :action => 'tex_list', :web => @web.address },
+             {'method' => 'post', 'accept-charset' => 'utf-8' }) do
+%>
+<div>
+<% if params['category'] -%>
+<input type="hidden" name="category" value="<%= params['category'] %>"/>
+<%- end -%>
+<ul id="sortable_pages">
+  <% @pages_in_category.each do |page| %>
+    <% content_tag_for :li, page do %>
+    
+      <input type="checkbox" name="<%= page.name %>" value="tex"/>       
+      <%= link_to_existing_page page, truncate(page.plain_name, :length => 35) %>
+    <% end %>
+
+  <% end %>
+</ul>
+<%= sortable_element('sortable_pages', {:onUpdate => 'function(){}'}) %>
+<label for="commit"> Export selected pages (drag to re-order them) to a LaTeX file.</label>
+<%= submit_tag("Export") %>
+</div>
+<%- end -%>
+<%- else -%>
 <ul>
   <%- @pages_in_category.each do |page| -%>
     <li>
@@ -17,6 +41,7 @@
     </li>
   <%- end -%>
 </ul>
+<%- end -%>
 
 <%- if @web.count_pages? -%>
   <% total_chars = @pages_in_category.characters %>
@@ -30,7 +55,7 @@
     Wanted Pages
     <br/>
     <span class="pageType">
-      Nonexistent pages that other pages in <%= @set_name %> reference
+      Nonexistent pages that other pages in <%= raw @set_name %> reference
     </span>
   </h2>
 
@@ -41,7 +66,7 @@
       wanted by
       <%= @web.select.pages_that_reference(wanted_page_name).collect { |referring_page|            
             link_to_existing_page referring_page
-          }.join(", ") 
+          }.join(", ").html_safe 
       %>
     </li>
   <%- end -%>
@@ -51,7 +76,7 @@
 <%- unless @pages_that_are_orphaned.empty? -%>
   <h2>
     Orphaned Pages
-    <br/><span class="pageType">Pages in <%= @set_name %> that no other page reference</span>
+    <br/><span class="pageType">Pages in <%= raw @set_name %> that no other page reference</span>
   </h2>
 
   <ul style="margin-bottom: 35px">

app/views/wiki/locked.rhtml

@@ -1,4 +1,4 @@
-<%- @title = "#{@page.plain_name} is locked" -%>
+<%- @title = "#{@page.plain_name} is locked".html_safe -%>
 
 <p>
   <%= link_to_page(h(@page.locked_by.purify)) %>

app/views/wiki/new.rhtml

@@ -1,12 +1,12 @@
 <%- 
-  @title = "Creating #{CGI.escapeHTML(WikiWords.separate(@page_name))}"
+  @title = "Creating #{WikiWords.separate(@page_name).escapeHTML}".html_safe
   @content_width = 720
   @hide_navigation = true
 -%>
 
 <div id="MarkupHelp">
-  <%= render(:file => "#{@web.markup}_help") %>
-  <%= render(:file => 'wiki_words_help') %>
+<%= render(:file => "#{@web.markup}_help") -%>
+<%= render(:file => 'wiki_words_help') unless @web.brackets_only? -%>
 </div>
 
 <div id="editForm">
@@ -14,7 +14,7 @@
 	        { 'id' => 'editForm', 'method' => 'post', 'onsubmit' => 'cleanAuthorName();', 'accept-charset' => 'utf-8' }) do %>
 	
 	    <textarea name="content" id="content" rows="24" cols="60"><%= h(flash[:content] || 
-                     ( (params['content'] && params['content'].is_utf8?) ?  params['content'] : '').purify ) %></textarea>
+                     params['content'] ?  params['content'].purify : '' ) %></textarea>
 		<div id="editFormButtons">
 		    <input type="submit" value="Submit" accesskey="s"/> as 
 		    <%= text_field_tag :author, @author, 
@@ -31,4 +31,8 @@ function cleanAuthorName() {
   }
 }
 document.forms["editForm"].elements["content"].focus();
+<%- if [:markdownMML, :markdownPNG, :markdown].include?(@web.markup) -%>
+setupSVGedit('<%= compute_public_path("editor/svg-editor.html", "svg-edit").split(/\?/)[0] %>');
+addS5button('<%= @page_name.escapeHTML %>');
+<%- end -%>
 </script>

app/views/wiki/page.rhtml

@@ -1,13 +1,13 @@
 <%- 
   @title = @page.plain_name
-  @title += ' (changes)' if @show_diff 
+  @title += ' (changes)' if @show_diff
   @show_footer = true
 -%>
 
 <div id="revision">
   <%- if @show_diff -%>
     <p class="show_diff">  
-        Showing changes from revision #<%= @page.revisions.size - 1 %> to #<%= @page.revisions.size %>:  
+        Showing changes from revision #<%= @page.rev_ids.size - 1 %> to #<%= @page.rev_ids.size %>:  
         <ins class="diffins">Added</ins> | <del class="diffdel">Removed</del> | <del class="diffmod">Chan</del><ins class="diffmod">ged</ins>  
     </p>
     <%= @renderer.display_diff %>
@@ -18,7 +18,7 @@
 
 <div class="byline">
   <%= @page.revisions? ? "Revised" : "Created" %> on <%= format_date(@page.revised_at) %> 
-  by <%= author_link(@page).purify %>
+  by <%= author_link(@page) %>
   <%= "(#{@page.author.ip})" if @page.author.respond_to?(:ip) %>
   <% if @web.count_pages? %>
     <% total_chars = @page.content.length %>
@@ -28,7 +28,7 @@
 
 <div class="navigation navfoot">
 
-  <%= navigation_menu_for_page.join(' | ') %>
+  <%= raw navigation_menu_for_page.join(' | ') %>
   
   <span class="views">
     | Views: 
@@ -45,6 +45,9 @@
             {:id => 'view_S5'} %>
    <%- end -%>
   <%- end -%>
+    |
+    <%= link_to 'Source', {:web => @web.address, :action => 'source', :id => @page.name}, 
+            {:id => 'view_source', :rel => 'nofollow' } %>
   </span>
 
   <%= render :partial => 'inbound_links' %>

app/views/wiki/print.rhtml

@@ -2,10 +2,9 @@
   @title = @page.plain_name
   @hide_navigation = true
   @style_additions = ".newWikiWord { background-color: white; font-style: italic; }"
-  @inline_style = true
 -%>
 
-<%= @renderer.display_content_for_export %>
+<%= @renderer.display_content %>
 
 <div class="byline">
   <%= @page.revisions? ? "Revised" : "Created" %> on <%= format_date(@page.revised_at) %> 

app/views/wiki/recently_revised.rhtml

@@ -8,9 +8,9 @@
   <%- for page in @pages_by_day[day] -%>
     <li>
       <%= link_to_existing_page page %>
-      <%- if page.revisions.size > 1 %>
+      <%- if page.rev_ids.size > 1 %>
           <span class="views">
-          ( <%= link_to_revision(page, page.revisions.size, text='diff',
+          ( <%= link_to_revision(page, page.rev_ids.size, text='diff',
                mode='diff') %> | <%= link_to_history(page, text='history') %> )
           </span>
       <%- end -%>

app/views/wiki/revision.rhtml

@@ -1,5 +1,5 @@
 <%- 
-  @title = "#{@page.plain_name} (Rev ##{@revision_number}#{@show_diff ? ', changes' : ''})"
+  @title = "#{@page.plain_name} (Rev ##{@revision_number}#{@show_diff ? ', changes' : ''})".html_safe
 -%>
 
 
@@ -21,6 +21,11 @@
 </div>
 
 <div class="navigation navfoot">
-  <%= navigation_menu_for_revision.join(' | ') %>
+  <%= raw navigation_menu_for_revision.join(' | ') %>
+    <span class="views">
+    | View: 
+    <%= link_to 'Source', {:web => @web.address, :action => 'source', :id => @page.name, :rev => @revision_number}, 
+            {:id => 'view_source', :rel => 'nofollow' } %>
+  </span>
   <%= render :partial => 'inbound_links' %>
 </div>

app/views/wiki/rollback.rhtml

@@ -1,14 +1,12 @@
 <%- 
-  @title = "Rollback to #{@page.plain_name} Rev ##{@revision_number}"
+  @title = "Rollback to #{@page.plain_name} Rev ##{@revision_number}".html_safe
   @content_width = 720
   @hide_navigation = true
 -%>
 
-<%= "<p style='color:red'>Please correct the error that caused this error in rendering:<br/><small>#{params["msg"]}</small></p>" if params["msg"] %>
-
 <div id="MarkupHelp">
-  <%= render(:file => "#{@web.markup}_help") %>
-  <%= render(:file => 'wiki_words_help') %>
+<%= render(:file => "#{@web.markup}_help") -%>
+<%= render(:file => 'wiki_words_help') unless @web.brackets_only? -%>
 </div>
 
 	<% form_tag({:web => @web.address, :action => 'save', :id => @page.name},

app/views/wiki/s5.rhtml

@@ -1,5 +1,4 @@
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1 plus MathML 2.0 plus SVG 1.1//EN" "http://www.w3.org/2002/04/xhtml-math-svg/xhtml-math-svg-flat.dtd" >
-
 <html xmlns="http://www.w3.org/1999/xhtml">
 
 <head>
@@ -13,15 +12,33 @@
 <meta name="defaultView" content="slideshow" />
 <meta name="controlVis" content="hidden" />
 <!-- style sheet links -->
-<link rel="stylesheet" href="/s5/ui/core/outline.css" type="text/css" media="screen" id="outlineStyle" />
-<link rel="stylesheet" href="/s5/ui/core/print.css" type="text/css" media="print" id="slidePrint" />
-<link rel="stylesheet" href="/s5/ui/core/opera.css" type="text/css" media="projection" id="operaFix" />
-<link rel="stylesheet" href="/s5/ui/core/math.css" type="text/css" media="all" id="mathStyle" />
-<link rel="stylesheet" href="/s5/themes/<%=@s5_theme%>/slides.css" type="text/css" media="projection" id="slideProj" />
+<%= stylesheet_link_tag "/stylesheets/syntax.css", :media => 'all', :id => 'syntaxStyle' %>
+<%= stylesheet_link_tag "/s5/ui/core/outline.css", :media => 'screen', :id => 'outlineStyle' %>
+<%= stylesheet_link_tag "/s5/ui/core/print.css", :media => 'print', :id => 'slidePrint' %>
+<%= stylesheet_link_tag "/s5/ui/core/opera.css", :media => 'projection', :id =>  'operaFix' %>
+<%= stylesheet_link_tag "/s5/ui/core/math.css", :media => 'all', :id =>  'mathStyle' %>
+<%= stylesheet_link_tag "/s5/themes/#{@s5_theme}/slides.css", :media => 'projection', :id =>  'slideProj' %>
 
 <!-- S5 JS -->
-<script src="/s5/ui/core/slides.js" type="text/javascript"></script>
-<script src="/javascripts/prototype.js" type="text/javascript"></script>
+<%= javascript_include_tag 'prototype' %>
+<%- if @web.markup == :markdownMML -%>
+<script type="text/x-mathjax-config">
+  MathJax.Hub.Config({
+    MathML: { useMathMLspacing: true },
+    "HTML-CSS": { scale: 90 }
+  });
+  if (window._onload_fired_) MathJax.Hub.Startup.onload();
+</script>
+<script type="text/javascript">
+  if (!(Prototype.Browser.Gecko || navigator.userAgent.match(/MathPlayer/))) {
+    var s = document.createElement('script');
+    s.src = "<%= compute_public_path('MathJax.js', 'MathJax').split('?')[0] %>?config=MML_HTMLorMML";
+    document.querySelector('head').appendChild(s);        
+    window.addEventListener("load", function(){window._onload_fired_ = true} , false);        
+  };
+</script>
+<%- end -%>
+<%= javascript_include_tag "/s5/ui/core/slides.js" %>
 </head>
 <body>
 

app/views/wiki/search.rhtml

@@ -1,4 +1,4 @@
-<%- @title = "Search results for \"#{h params["query"]}\"" -%>
+<%- @title = "Search results for \"#{h @query}\"".html_safe -%>
 
 <%- unless @title_results.empty? -%>
 <h2><%= @title_results.length %> page(s) containing search string in the page name:</h2>
@@ -24,7 +24,7 @@
 <%- end -%>
 
 <%- if (@results + @title_results).empty? -%>
-  <h2>No pages contain "<%= h params["query"] %>" </h2>
+  <h2>No pages contain "<%= h(@query).html_safe %>" </h2>
   <p>
     Perhaps you should try expanding your query. Remember that Instiki searches for entire 
     phrases, so if you search for "all that jazz" it will not match pages that contain these 
@@ -36,6 +36,6 @@
     "[a-z]*Leet?RegExpSkill(s|z)"
   </p>
   <p>
-    <b>Create a new page, named:</b> "<span class='newWikiWord'><%= link_to h(params[:query]), :web => @web.address, :action => 'new', :id => params[:query] %></span>"
+    <b>Create a new page, named:</b> "<span class='newWikiWord'><%= link_to h(@query).html_safe, :web => @web.address, :action => 'new', :id => @query %></span>"
   </p>
 <%- end -%>

app/views/wiki/source.html.erb

@@ -0,0 +1,64 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1 plus MathML 2.0 plus SVG 1.1//EN" "http://www.w3.org/2002/04/xhtml-math-svg/xhtml-math-svg-flat.dtd" >
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+  <title>
+    <%- if @page and (@page.name == 'HomePage') -%>
+      <%= h(@web.name) %>
+    <%- else @web -%>
+      <%= @page.plain_name %> in <%= h @web.name %>
+    <%- end -%>
+  </title>
+
+  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+  <meta name="robots" content="<%= @robots_metatag_value %>" />
+  
+  <style type="text/css">
+    h1#pageName, div.info, .newWikiWord a, a.existingWikiWord, .newWikiWord a:hover, [actiontype="toggle"]:hover, #TextileHelp h3 { 
+      color: #<%= @web ? @web.color : "393" %>; 
+    }
+    <%= Rails.root.join('public', 'stylesheets', 'instiki.css').read if @inline_style %>
+  </style>
+  <%= stylesheet_link_tag 'instiki', :media => 'all'  unless @inline_style %>
+  <%= "<style type='text/css'>#{@style_additions}</style>".html_safe if @style_additions %>
+  <style type="text/css"><!--/*--><![CDATA[/*><!--*/    
+    <%= @web ? @web.additional_style : '' %>
+  /*]]>*/--></style>
+  <%= javascript_include_tag :defaults %>
+<script type="text/javascript">
+<!--//--><![CDATA[//><!--
+function updateSize(elt, w, h) {
+   // adjust to the size of the user's browser area.
+   // w and h are the original, unadjusted, width and height per row/column
+    var parentheight = document.viewport.getHeight();
+    var parentwidth = $('Container').getWidth();
+    elt.writeAttribute({'cols': Math.floor(parentwidth/w)  - 1,
+                        'rows': Math.floor(parentheight/h) - 2 });
+    elt.setStyle({Width: parentwidth, Height: parentheight});
+}
+
+function resizeableTextarea() {
+//make the textarea resize to fit available space
+  $$('textarea#content').each( function(textarea)  {
+    var w = textarea.getWidth()/textarea.getAttribute('cols');
+    var h = textarea.getStyle('lineHeight').replace(/(\d*)px/, "$1");
+    Event.observe(window, 'resize', function(){ updateSize(textarea, w, h) });
+    updateSize(textarea, w, h);
+    Form.Element.focus(textarea);
+  });
+}
+
+window.onload = function (){
+        resizeableTextarea();
+}
+//--><!]]>
+</script>
+</head>
+
+<body>
+
+<div id="Container">
+<textarea id='content'  readonly=' readonly' rows='24' cols='60' ><%= (@revision ? @revision.content : @page.content).purify %></textarea>
+</div> <!-- Container -->
+
+</body>
+</html>

app/views/wiki/tex.rhtml

@@ -1,239 +1,3 @@
-\documentclass[12pt,titlepage]{article}
-
-\usepackage{amsmath}
-\usepackage{amsfonts}
-\usepackage{amssymb}
-\usepackage{amsthm}
-<%- if @tex_content =~  /\\mathbb\{[^}]*[a-z0-9]+[^}]*\}/ -%>
-\usepackage{mathbbol}
-<% end -%>
-\usepackage{graphicx}
-\usepackage{color}
-\usepackage{ucs}
-\usepackage[utf8x]{inputenc}
-\usepackage{hyperref}
-
-%----Macros----------
-%
-% Unresolved issues:
-%
-%  \righttoleftarrow
-%  \lefttorightarrow
-%
-%  \color{} with HTML colorspec
-%  \bgcolor
-%  \array
-
-% Of the standard HTML named colors, white, black, red, green, blue and yellow
-% are predefined in the color package. Here are the rest.
-\definecolor{aqua}{rgb}{0, 1.0, 1.0}
-\definecolor{fuschia}{rgb}{1.0, 0, 1.0}
-\definecolor{gray}{rgb}{0.502, 0.502, 0.502}
-\definecolor{lime}{rgb}{0, 1.0, 0}
-\definecolor{maroon}{rgb}{0.502, 0, 0}
-\definecolor{navy}{rgb}{0, 0, 0.502}
-\definecolor{olive}{rgb}{0.502, 0.502, 0}
-\definecolor{purple}{rgb}{0.502, 0, 0.502}
-\definecolor{silver}{rgb}{0.753, 0.753, 0.753}
-\definecolor{teal}{rgb}{0, 0.502, 0.502}
-
-% Because of conflicts, \space and \mathop are converted to
-% \itexspace and \operatorname during preprocessing.
-
-% itex: \space{ht}{dp}{wd}
-%
-% Height and baseline depth measurements are in units of tenths of an ex while
-% the width is measured in tenths of an em.
-\makeatletter
-\newdimen\itex@wd%
-\newdimen\itex@dp%
-\newdimen\itex@thd%
-\def\itexspace#1#2#3{\itex@wd=#3em%
-\itex@wd=0.1\itex@wd%
-\itex@dp=#2ex%
-\itex@dp=0.1\itex@dp%
-\itex@thd=#1ex%
-\itex@thd=0.1\itex@thd%
-\advance\itex@thd\the\itex@dp%
-\makebox[\the\itex@wd]{\rule[-\the\itex@dp]{0cm}{\the\itex@thd}}}
-\makeatother
-
-% \tensor and \multiscript
-\makeatletter
-\newif\if@sup
-\newtoks\@sups
-\def\append@sup#1{\edef\act{\noexpand\@sups={\the\@sups #1}}\act}%
-\def\reset@sup{\@supfalse\@sups={}}%
-\def\mk@scripts#1#2{\if #2/ \if@sup ^{\the\@sups}\fi \else%
-  \ifx #1_ \if@sup ^{\the\@sups}\reset@sup \fi {}_{#2}%
-  \else \append@sup#2 \@suptrue \fi%
-  \expandafter\mk@scripts\fi}
-\def\tensor#1#2{\reset@sup#1\mk@scripts#2_/}
-\def\multiscripts#1#2#3{\reset@sup{}\mk@scripts#1_/#2%
-  \reset@sup\mk@scripts#3_/}
-\makeatother
-
-% \slash
-\makeatletter
-\newbox\slashbox \setbox\slashbox=\hbox{$/$}
-\def\itex@pslash#1{\setbox\@tempboxa=\hbox{$#1$}
-  \@tempdima=0.5\wd\slashbox \advance\@tempdima 0.5\wd\@tempboxa
-  \copy\slashbox \kern-\@tempdima \box\@tempboxa}
-\def\slash{\protect\itex@pslash}
-\makeatother
-
-% Renames \sqrt as \oldsqrt and redefine root to result in \sqrt[#1]{#2}
-\let\oldroot\root
-\def\root#1#2{\oldroot #1 \of{#2}}
-
-% Manually declare the txfonts symbolsC font
-\DeclareSymbolFont{symbolsC}{U}{txsyc}{m}{n}
-\SetSymbolFont{symbolsC}{bold}{U}{txsyc}{bx}{n}
-\DeclareFontSubstitution{U}{txsyc}{m}{n}
-
-% Manually declare the stmaryrd font
-\DeclareSymbolFont{stmry}{U}{stmry}{m}{n}
-\SetSymbolFont{stmry}{bold}{U}{stmry}{b}{n}
-
-% Declare specific arrows from txfonts without loading the full package
-\makeatletter
-\def\re@DeclareMathSymbol#1#2#3#4{%
-    \let#1=\undefined
-    \DeclareMathSymbol{#1}{#2}{#3}{#4}}
-\re@DeclareMathSymbol{\neArrow}{\mathrel}{symbolsC}{116}
-\re@DeclareMathSymbol{\neArr}{\mathrel}{symbolsC}{116}
-\re@DeclareMathSymbol{\seArrow}{\mathrel}{symbolsC}{117}
-\re@DeclareMathSymbol{\seArr}{\mathrel}{symbolsC}{117}
-\re@DeclareMathSymbol{\nwArrow}{\mathrel}{symbolsC}{118}
-\re@DeclareMathSymbol{\nwArr}{\mathrel}{symbolsC}{118}
-\re@DeclareMathSymbol{\swArrow}{\mathrel}{symbolsC}{119}
-\re@DeclareMathSymbol{\swArr}{\mathrel}{symbolsC}{119}
-\re@DeclareMathSymbol{\nequiv}{\mathrel}{symbolsC}{46}
-\re@DeclareMathSymbol{\Perp}{\mathrel}{symbolsC}{121}
-\re@DeclareMathSymbol{\Vbar}{\mathrel}{symbolsC}{121}
-\re@DeclareMathSymbol{\sslash}{\mathrel}{stmry}{12}
-\re@DeclareMathSymbol{\invamp}{\mathrel}{symbolsC}{77}
-\re@DeclareMathSymbol{\parr}{\mathrel}{symbolsC}{77}
-\makeatother
-
-% Widecheck
-\makeatletter
-\DeclareRobustCommand\widecheck[1]{{\mathpalette\@widecheck{#1}}}
-\def\@widecheck#1#2{%
-    \setbox\z@\hbox{\m@th$#1#2$}%
-    \setbox\tw@\hbox{\m@th$#1%
-       \widehat{%
-          \vrule\@width\z@\@height\ht\z@
-          \vrule\@height\z@\@width\wd\z@}$}%
-    \dp\tw@-\ht\z@
-    \@tempdima\ht\z@ \advance\@tempdima2\ht\tw@ \divide\@tempdima\thr@@
-    \setbox\tw@\hbox{%
-       \raise\@tempdima\hbox{\scalebox{1}[-1]{\lower\@tempdima\box
-\tw@}}}%
-    {\ooalign{\box\tw@ \cr \box\z@}}}
-\makeatother
-
-% udots (taken from yhmath)
-\makeatletter
-\def\udots{\mathinner{\mkern2mu\raise\p@\hbox{.}
-\mkern2mu\raise4\p@\hbox{.}\mkern1mu
-\raise7\p@\vbox{\kern7\p@\hbox{.}}\mkern1mu}}
-\makeatother
-
-%% Renaming existing commands
-\newcommand{\underoverset}[3]{\underset{#1}{\overset{#2}{#3}}}
-\newcommand{\widevec}{\overrightarrow}
-\newcommand{\darr}{\downarrow}
-\newcommand{\nearr}{\nearrow}
-\newcommand{\nwarr}{\nwarrow}
-\newcommand{\searr}{\searrow}
-\newcommand{\swarr}{\swarrow}
-\newcommand{\curvearrowbotright}{\curvearrowright}
-\newcommand{\uparr}{\uparrow}
-\newcommand{\downuparrow}{\updownarrow}
-\newcommand{\duparr}{\updownarrow}
-\newcommand{\updarr}{\updownarrow}
-\newcommand{\gt}{>}
-\newcommand{\lt}{<}
-\newcommand{\map}{\mapsto}
-\newcommand{\embedsin}{\hookrightarrow}
-\newcommand{\Alpha}{A}
-\newcommand{\Beta}{B}
-\newcommand{\Zeta}{Z}
-\newcommand{\Eta}{H}
-\newcommand{\Iota}{I}
-\newcommand{\Kappa}{K}
-\newcommand{\Mu}{M}
-\newcommand{\Nu}{N}
-\newcommand{\Rho}{P}
-\newcommand{\Tau}{T}
-\newcommand{\Upsi}{\Upsilon}
-\newcommand{\omicron}{o}
-\newcommand{\lang}{\langle}
-\newcommand{\rang}{\rangle}
-\newcommand{\Union}{\bigcup}
-\newcommand{\Intersection}{\bigcap}
-\newcommand{\Oplus}{\bigoplus}
-\newcommand{\Otimes}{\bigotimes}
-\newcommand{\Wedge}{\bigwedge}
-\newcommand{\Vee}{\bigvee}
-\newcommand{\coproduct}{\coprod}
-\newcommand{\product}{\prod}
-\newcommand{\closure}{\overline}
-\newcommand{\integral}{\int}
-\newcommand{\doubleintegral}{\iint}
-\newcommand{\tripleintegral}{\iiint}
-\newcommand{\quadrupleintegral}{\iiiint}
-\newcommand{\conint}{\oint}
-\newcommand{\contourintegral}{\oint}
-\newcommand{\infinity}{\infty}
-\renewcommand{\empty}{\emptyset}
-\newcommand{\bottom}{\bot}
-\newcommand{\minusb}{\boxminus}
-\newcommand{\plusb}{\boxplus}
-\newcommand{\timesb}{\boxtimes}
-\newcommand{\intersection}{\cap}
-\newcommand{\union}{\cup}
-\newcommand{\Del}{\nabla}
-\newcommand{\odash}{\circleddash}
-\newcommand{\negspace}{\!}
-\newcommand{\widebar}{\overline}
-\newcommand{\textsize}{\normalsize}
-\renewcommand{\scriptsize}{\scriptstyle}
-\newcommand{\scriptscriptsize}{\scriptscriptstyle}
-\newcommand{\mathfr}{\mathfrak}
-\newcommand{\statusline}[2]{#2}
-\newcommand{\toggle}[2]{#1}
-
-% Theorem Environments
-\theoremstyle{plain}
-\newtheorem{theorem}{Theorem}
-\newtheorem{lemma}{Lemma}
-\newtheorem{prop}{Proposition}
-\newtheorem{cor}{Corollary}
-\newtheorem*{utheorem}{Theorem}
-\newtheorem*{ulemma}{Lemma}
-\newtheorem*{uprop}{Proposition}
-\newtheorem*{ucor}{Corollary}
-\theoremstyle{definition}
-\newtheorem{defn}{Definition}
-\newtheorem{example}{Example}
-\newtheorem*{udefn}{Definition}
-\newtheorem*{uexample}{Example}
-\theoremstyle{remark}
-\newtheorem{remark}{Remark}
-\newtheorem{note}{Note}
-\newtheorem*{uremark}{Remark}
-\newtheorem*{unote}{Note}
-
-%-------------------------------------------------------------------
-
-\begin{document}
-
-%-------------------------------------------------------------------
-
 \section*{<%= @page.name %>}
 
-<%= @tex_content %>
-
-\end{document}
+<%= @tex_content.html_safe %>

app/views/wiki/tex_list.html.erb

@@ -0,0 +1 @@
+<%= @tex_content.html_safe %>

app/views/wiki/web_list.rhtml

@@ -19,7 +19,7 @@
 	- Last Update: <%= web.last_page.nil? ? format_date(web.created_at) : format_date(web.last_page.revised_at) %><br/>
 	<%- if ! web.last_page.nil? -%> 
         Last Document: <%= link_to_page(web.last_page.name,web) %> 
-        <%= web.last_page.revisions? ? "Revised" : "Created" %> by <%= author_link(web.last_page).purify %> (<%= web.last_page.current_revision.ip %>)
+        <%= web.last_page.revisions? ? "Revised" : "Created" %> by <%= author_link(web.last_page) %> (<%= web.last_page.current_revision.ip %>)
 	<%- end -%>
     </div>
   </div>

attic/lib/sanitize.rb

@@ -1,207 +1,262 @@
+# == Introduction
+#
+# This module provides sanitization of XHTML+MathML+SVG 
+# and of inline style attributes. Its genesis is {described here}[http://golem.ph.utexas.edu/~distler/blog/archives/001181.html].
+#
+# Uses the {HTML5lib parser}[http://code.google.com/p/html5lib/], so that the parsing behaviour should
+# resemble that of browsers.
+#
+#  sanitize_xhtml() is a case-sensitive sanitizer, suitable for XHTML
+#  sanitize_html() is a case-insensitive sanitizer suitable for HTML
+#  sanitize_rexml() sanitizes a REXML tree, returning a string
+#  safe_sanitize_xhtml() makes extra-sure that the result is well-formed XHTML
+#                        by running the output of sanitize_xhtml() through REXML
+#
+# == Files
+#
+# {sanitize.rb}[http://golem.ph.utexas.edu/~distler/code/instiki/svn/lib/sanitize.rb],
+# {HTML5lib}[http://golem.ph.utexas.edu/~distler/code/instiki/svn/vendor/plugins/HTML5lib/]
+#
+# == Author
+#
+# {Jacques Distler}[http://golem.ph.utexas.edu/~distler/]
+#
+# ==  License
+#
+# Ruby License
+
 module Sanitize
 
-# This module provides sanitization of XHTML+MathML+SVG
-# and of inline style attributes.
+  require 'html5/html5parser'
+  require 'html5/liberalxmlparser'
+  require 'html5/treewalkers'
+  require 'html5/treebuilders'
+  require 'html5/serializer'
+  require 'html5/sanitizer'
+  require 'stringsupport.rb'
+
+  include HTML5
+
+# Sanitize a string, parsed using XHTML parsing rules.
 #
-# Based heavily on Sam Ruby's code in the Universal FeedParser.
-
-  require 'html/tokenizer'
-  require 'node'
-
-  acceptable_elements = ['a', 'abbr', 'acronym', 'address', 'area', 'b',
-      'big', 'blockquote', 'br', 'button', 'caption', 'center', 'cite',
-      'code', 'col', 'colgroup', 'dd', 'del', 'dfn', 'dir', 'div', 'dl', 'dt',
-      'em', 'fieldset', 'font', 'form', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6',
-      'hr', 'i', 'img', 'input', 'ins', 'kbd', 'label', 'legend', 'li', 'map',
-      'menu', 'ol', 'optgroup', 'option', 'p', 'pre', 'q', 's', 'samp',
-      'select', 'small', 'span', 'strike', 'strong', 'sub', 'sup', 'table',
-      'tbody', 'td', 'textarea', 'tfoot', 'th', 'thead', 'tr', 'tt', 'u',
-      'ul', 'var']
-      
-  mathml_elements = ['maction', 'math', 'merror', 'mfrac', 'mi',
-      'mmultiscripts', 'mn', 'mo', 'mover', 'mpadded', 'mphantom',
-      'mprescripts', 'mroot', 'mrow', 'mspace', 'msqrt', 'mstyle', 'msub',
-      'msubsup', 'msup', 'mtable', 'mtd', 'mtext', 'mtr', 'munder',
-      'munderover', 'none']
-      
-  svg_elements = ['a', 'animate', 'animateColor', 'animateMotion',
-      'animateTransform', 'circle', 'defs', 'desc', 'ellipse', 'font-face',
-      'font-face-name', 'font-face-src', 'g', 'glyph', 'hkern', 'image',
-      'linearGradient', 'line', 'marker', 'metadata', 'missing-glyph',
-      'mpath', 'path', 'polygon', 'polyline', 'radialGradient', 'rect',
-      'set', 'stop', 'svg', 'switch', 'text', 'title', 'tspan', 'use']
-      
-  acceptable_attributes = ['abbr', 'accept', 'accept-charset', 'accesskey',
-      'action', 'align', 'alt', 'axis', 'border', 'cellpadding',
-      'cellspacing', 'char', 'charoff', 'charset', 'checked', 'cite', 'class',
-      'clear', 'cols', 'colspan', 'color', 'compact', 'coords', 'datetime',
-      'dir', 'disabled', 'enctype', 'for', 'frame', 'headers', 'height',
-      'href', 'hreflang', 'hspace', 'id', 'ismap', 'label', 'lang',
-      'longdesc', 'maxlength', 'media', 'method', 'multiple', 'name',
-      'nohref', 'noshade', 'nowrap', 'prompt', 'readonly', 'rel', 'rev',
-      'rows', 'rowspan', 'rules', 'scope', 'selected', 'shape', 'size',
-      'span', 'src', 'start', 'style', 'summary', 'tabindex', 'target', 'title',
-      'type', 'usemap', 'valign', 'value', 'vspace', 'width', 'xml:lang']
-
-
-  mathml_attributes = ['actiontype', 'align', 'columnalign', 'columnalign',
-      'columnalign', 'columnlines', 'columnspacing', 'columnspan', 'depth',
-      'display', 'displaystyle', 'equalcolumns', 'equalrows', 'fence',
-      'fontstyle', 'fontweight', 'frame', 'height', 'linethickness', 'lspace',
-      'mathbackground', 'mathcolor', 'mathvariant', 'mathvariant', 'maxsize',
-      'minsize', 'other', 'rowalign', 'rowalign', 'rowalign', 'rowlines',
-      'rowspacing', 'rowspan', 'rspace', 'scriptlevel', 'selection',
-      'separator', 'stretchy', 'width', 'width', 'xlink:href', 'xlink:show',
-      'xlink:type', 'xmlns', 'xmlns:xlink']
-
-      
-  svg_attributes = ['accent-height', 'accumulate', 'additive', 'alphabetic',
-       'arabic-form', 'ascent', 'attributeName', 'attributeType',
-       'baseProfile', 'bbox', 'begin', 'by', 'calcMode', 'cap-height',
-       'class', 'color', 'color-rendering', 'content', 'cx', 'cy', 'd', 'dx',
-       'dy', 'descent', 'display', 'dur', 'end', 'fill', 'fill-rule',
-       'font-family', 'font-size', 'font-stretch', 'font-style', 'font-variant',
-       'font-weight', 'from', 'fx', 'fy', 'g1', 'g2', 'glyph-name', 
-       'gradientUnits', 'hanging', 'height', 'horiz-adv-x', 'horiz-origin-x',
-       'id', 'ideographic', 'k', 'keyPoints', 'keySplines', 'keyTimes',
-       'lang', 'marker-end', 'marker-mid', 'marker-start', 'markerHeight',
-       'markerUnits', 'markerWidth', 'mathematical', 'max', 'min', 'name',
-       'offset', 'opacity', 'orient', 'origin', 'overline-position',
-       'overline-thickness', 'panose-1', 'path', 'pathLength', 'points',
-       'preserveAspectRatio', 'r', 'refX', 'refY', 'repeatCount', 'repeatDur',
-       'requiredExtensions', 'requiredFeatures', 'restart', 'rotate', 'rx',
-       'ry', 'slope', 'stemh', 'stemv', 'stop-color', 'stop-opacity',
-       'strikethrough-position', 'strikethrough-thickness', 'stroke',
-       'stroke-dasharray', 'stroke-dashoffset', 'stroke-linecap',
-       'stroke-linejoin', 'stroke-miterlimit', 'stroke-opacity',
-       'stroke-width', 'systemLanguage', 'target',
-       'text-anchor', 'to', 'transform', 'type', 'u1', 'u2',
-       'underline-position', 'underline-thickness', 'unicode',
-       'unicode-range', 'units-per-em', 'values', 'version', 'viewBox',
-       'visibility', 'width', 'widths', 'x', 'x-height', 'x1', 'x2',
-       'xlink:actuate', 'xlink:arcrole', 'xlink:href', 'xlink:role',
-       'xlink:show', 'xlink:title', 'xlink:type', 'xml:base', 'xml:lang',
-       'xml:space', 'xmlns', 'xmlns:xlink', 'y', 'y1', 'y2', 'zoomAndPan']
-
-  attr_val_is_uri = ['href', 'src', 'cite', 'action', 'longdesc', 'xlink:href']
+# :call-seq:
+#    sanitize_xhtml(string)                    -> string
+#    sanitize_xhtml(string, {:encoding => 'iso-8859-1', :to_tree => true}) -> REXML::Document
+#
+# Unless otherwise specified, the string is assumed to be utf-8 encoded.
+# By default, the output is a string. But, optionally, you can return a REXML tree.
+#
+# The string returned is utf-8 encoded. If you want, you can use iconv to convert it to some other encoding.
+# (REXML trees are always utf-8 encoded.)
+  def sanitize_xhtml(html, options = {})
+    @encoding = 'utf-8'
+    @treebuilder = TreeBuilders::REXML::TreeBuilder
+    @to_tree = false
+    options.each do |name, value|
+      next unless %w(encoding treebuilder to_tree).include? name.to_s
+      if name.to_s == 'treebuilder'
+        @treebuilder =  HTML5lib::TreeBuilders.get_tree_builder(value)
+      else
+        instance_variable_set("@#{name}", value)
+      end
+    end
+    if @encoding == 'utf-8'
+      parsed = XHTMLParser.parse_fragment(html.to_utf8, {:tokenizer => HTMLSanitizer,
+        :lowercase_element_name => false, :lowercase_attr_name => false,
+        :encoding => @encoding, :tree => @treebuilder })
+    else
+      parsed = XHTMLParser.parse_fragment(html.to_ncr, {:tokenizer => HTMLSanitizer,
+        :lowercase_element_name => false, :lowercase_attr_name => false,
+        :encoding => @encoding, :tree => @treebuilder })
+    end      
+    return parsed if @to_tree
+    return parsed.to_s
+  end
   
-  acceptable_css_properties = ['azimuth', 'background-color',
-      'border-bottom-color', 'border-collapse', 'border-color',
-      'border-left-color', 'border-right-color', 'border-top-color', 'clear',
-      'color', 'cursor', 'direction', 'display', 'elevation', 'float', 'font',
-      'font-family', 'font-size', 'font-style', 'font-variant', 'font-weight',
-      'height', 'letter-spacing', 'line-height', 'overflow', 'pause',
-      'pause-after', 'pause-before', 'pitch', 'pitch-range', 'richness',
-      'speak', 'speak-header', 'speak-numeral', 'speak-punctuation',
-      'speech-rate', 'stress', 'text-align', 'text-decoration', 'text-indent',
-      'unicode-bidi', 'vertical-align', 'voice-family', 'volume',
-      'white-space', 'width']
+# Sanitize a string, parsed using XHTML parsing rules. Reparse the result to
+#    ensure well-formedness. 
+#
+# :call-seq:
+#    safe_sanitize_xhtml(string)                    -> string
+#
+# Unless otherwise specified, the string is assumed to be utf-8 encoded.
+#
+# The string returned is utf-8 encoded. If you want, you can use iconv to convert it to some other encoding.
+# (REXML trees are always utf-8 encoded.)
+  def safe_sanitize_xhtml(html, options = {})
+    options[:to_tree] = false
+    sanitized = sanitize_xhtml(html, options)
+    doc = REXML::Document.new("<div xmlns='http://www.w3.org/1999/xhtml'>#{sanitized}</div>")
+    sanitized = doc.to_s.gsub(/\A<div xmlns='http:\/\/www.w3.org\/1999\/xhtml'>(.*)<\/div>\Z/m, '\1')
+    rescue REXML::ParseException
+      sanitized = sanitized.escapeHTML
+  end 
 
-  acceptable_css_keywords = ['auto', 'aqua', 'black', 'block', 'blue',
-      'bold', 'both', 'bottom', 'brown', 'center', 'collapse', 'dashed',
-      'dotted', 'fuchsia', 'gray', 'green', '!important', 'italic', 'left',
-      'lime', 'maroon', 'medium', 'none', 'navy', 'normal', 'nowrap', 'olive',
-      'pointer', 'purple', 'red', 'right', 'solid', 'silver', 'teal', 'top',
-      'transparent', 'underline', 'white', 'yellow']
+# Sanitize a string, parsed using HTML parsing rules.
+#
+# :call-seq:
+#    sanitize_html( string )                    ->  string
+#    sanitize_html( string, {:encoding => 'iso-8859-1', :to_tree => true} ) ->  REXML::Document
+#
+# Unless otherwise specified, the string is assumed to be utf-8 encoded.
+# By default, the output is a string. But, optionally, you can return a REXML tree.
+#
+# The string returned is utf-8 encoded. If you want, you can use iconv to convert it to some other encoding.
+# (REXML trees are always utf-8 encoded.)
+  def sanitize_html(html, options = {})
+    @encoding = 'utf-8'
+    @treebuilder = TreeBuilders::REXML::TreeBuilder
+    @to_tree = false
+    options.each do |name, value|
+      next unless %w(encoding treebuilder to_tree).include? name.to_s
+      if name.to_s == 'treebuilder'
+        @treebuilder =  HTML5lib::TreeBuilders.get_tree_builder(value)
+      else
+        instance_variable_set("@#{name}", value)
+      end
+    end
+    if @encoding == 'utf-8'
+      parsed = HTMLParser.parse_fragment(html.to_utf8, {:tokenizer => HTMLSanitizer,
+        :encoding => @encoding, :tree => @treebuilder })
+    else
+      parsed = HTMLParser.parse_fragment(html.to_ncr, {:tokenizer => HTMLSanitizer,
+        :encoding => @encoding, :tree => @treebuilder })
+    end 
+    return parsed if @to_tree
+    return parsed.to_s
+  end
 
-  acceptable_svg_properties = [ 'fill', 'fill-opacity', 'fill-rule',
-      'stroke', 'stroke-width', 'stroke-linecap', 'stroke-linejoin',
-      'stroke-opacity']
+# Sanitize a REXML tree. The output is a string.
+#
+# :call-seq:
+#    sanitize_rexml(tree)                    -> string
+#
+  def sanitize_rexml(tree)
+    tokens = TreeWalkers.get_tree_walker('rexml2').new(tree)
+    XHTMLSerializer.serialize(tokens, {:encoding=>'utf-8',
+      :space_before_trailing_solidus => true,
+      :inject_meta_charset => false,
+      :sanitize => true})
+  end
+end
 
-  acceptable_protocols = [ 'ed2k', 'ftp', 'http', 'https', 'irc',
-      'mailto', 'news', 'gopher', 'nntp', 'telnet', 'webcal',
-      'xmpp', 'callto', 'feed', 'urn', 'aim', 'rsync', 'tag',
-      'ssh', 'sftp', 'rtsp', 'afs' ]
+require 'rexml/element'
+module REXML #:nodoc:
+  class Element
 
-      ALLOWED_ELEMENTS = acceptable_elements + mathml_elements + svg_elements  unless defined?(ALLOWED_ELEMENTS)
-      ALLOWED_ATTRIBUTES = acceptable_attributes + mathml_attributes + svg_attributes unless defined?(ALLOWED_ATTRIBUTES)
-      ALLOWED_CSS_PROPERTIES = acceptable_css_properties unless defined?(ALLOWED_CSS_PROPERTIES)
-      ALLOWED_CSS_KEYWORDS = acceptable_css_keywords unless defined?(ALLOWED_CSS_KEYWORDS)
-      ALLOWED_SVG_PROPERTIES = acceptable_svg_properties unless defined?(ALLOWED_SVG_PROPERTIES)
-      ALLOWED_PROTOCOLS = acceptable_protocols unless defined?(ALLOWED_PROTOCOLS)
-      ATTR_VAL_IS_URI = attr_val_is_uri unless defined?(ATTR_VAL_IS_URI)
+# Convert XHTML+MathML Named Entities in a REXML::Element to Numeric Character References
+#
+#  :call-seq:
+#     tree.to_ncr  -> REXML::Element
+#
+# REXML, typically, converts NCRs to utf-8 characters, which is what you'll see when you
+# access the resulting REXML document.
+#
+# Note that this method needs to traverse the entire tree, converting text nodes and attributes
+# for each element. This can be SLOW. It will often be faster to serialize to a string and then
+# use String.to_ncr instead.
+#
+    def to_ncr
+      self.each_element { |el|
+        el.texts.each_index  {|i|
+          el.texts[i].value = el.texts[i].to_s.to_ncr
+        }
+        el.attributes.each { |name,val|
+          el.attributes[name] = val.to_ncr
+        }
+        el.to_ncr if el.has_elements?
+      }
+      return self
+    end
+    
+# Convert XHTML+MathML Named Entities in a REXML::Element to UTF-8
+#
+#  :call-seq:
+#     tree.to_utf8  -> REXML::Element
+#
+# Note that this method needs to traverse the entire tree, converting text nodes and attributes 
+# for each element. This can be SLOW. It will often be faster to serialize to a string and then
+# use String.to_utf8 instead.
+#
+    def to_utf8
+      self.each_element { |el|
+        el.texts.each_index  {|i|
+          el.texts[i].value = el.texts[i].to_s.to_utf8
+        }
+        el.attributes.each { |name,val|
+          el.attributes[name] = val.to_utf8
+        }
+        el.to_utf8 if el.has_elements?
+      }
+      return self
+    end
 
-      # Sanitize the +html+, escaping all elements not in ALLOWED_ELEMENTS, and stripping out all
-      # attributes not in ALLOWED_ATTRIBUTES. Style attributes are parsed, and a restricted set,
-      # specified by ALLOWED_CSS_PROPERTIES and ALLOWED_CSS_KEYWORDS, are allowed through.
-      # attributes in ATTR_VAL_IS_URI are scanned, and only URI schemes specified in
-      # ALLOWED_PROTOCOLS are allowed.
-      # You can adjust what gets sanitized, by defining these constant arrays before this Module is loaded. 
-      #
-      #   sanitize_html('<script> do_nasty_stuff() </script>')
-      #    => &lt;script> do_nasty_stuff() &lt;/script>
-      #   sanitize_html('<a href="javascript: sucker();">Click here for $100</a>')
-      #    => <a>Click here for $100</a>
-      def sanitize_html(html)
-        if html.index("<")
-          tokenizer = HTML::Tokenizer.new(html)
-          new_text = ""
+  end
+end
 
-          while token = tokenizer.next
-            node = XHTML::Node.parse(nil, 0, 0, token, false)
-            new_text << case node.tag?
-              when true
-                if ALLOWED_ELEMENTS.include?(node.name)
-                  if node.closing != :close
-                    node.attributes.delete_if { |attr,v| !ALLOWED_ATTRIBUTES.include?(attr) }
-                    ATTR_VAL_IS_URI.each do |attr|
-                      val_unescaped = CGI.unescapeHTML(node.attributes[attr].to_s).gsub(/[\000-\040\177\s]+|\302*[\200-\240]/,'').downcase
-                      if val_unescaped =~ /^[a-z0-9][-+.a-z0-9]*:/ and !ALLOWED_PROTOCOLS.include?(val_unescaped.split(':')[0]) 
-                        node.attributes.delete attr 
-                      end
-                    end
-                    if node.attributes['style']
-                      node.attributes['style'] = sanitize_css(node.attributes['style']) 
-                    end
-                  end
-                  node.to_s
-                else
-                  node.to_s.gsub(/</, "&lt;")
-                end
-              else
-                node.to_s.gsub(/</, "&lt;")
-            end
-          end
+module HTML5 #:nodoc: all
+  module TreeWalkers
 
-          html = new_text
+    private
+
+    class << self
+      def [](name)
+        case name.to_s.downcase
+        when 'rexml'
+          require 'html5/treewalkers/rexml'
+          REXML::TreeWalker
+        when 'rexml2'
+          REXML2::TreeWalker
+        else
+          raise "Unknown TreeWalker #{name}"
         end
-        html
       end
-      
-      def sanitize_css(style)
-          # disallow urls
-          style = style.to_s.gsub(/url\s*\(\s*[^\s)]+?\s*\)\s*/, ' ')
 
-          # gauntlet
-          if style !~ /^([:,;#%.\sa-zA-Z0-9!]|\w-\w|\'[\s\w]+\'|\"[\s\w]+\"|\([\d,\s]+\))*$/
-             style = ''
-             return style
-          end
-          if style !~ /^(\s*[-\w]+\s*:\s*[^:;]*(;|$))*$/
-             style = ''
-             return style
-          end
+      alias :get_tree_walker :[]
+    end
 
-          clean = []
-          style.scan(/([-\w]+)\s*:\s*([^:;]*)/) do |prop,val|
-            if ALLOWED_CSS_PROPERTIES.include?(prop.downcase)
-              clean <<  prop + ': ' + val + ';'
-            elsif ['background','border','margin','padding'].include?(prop.split('-')[0].downcase) 
-              goodval = true
-              val.split().each do |keyword|
-                if !ALLOWED_CSS_KEYWORDS.include?(keyword) and 
-                   keyword !~ /^(#[0-9a-f]+|rgb\(\d+%?,\d*%?,?\d*%?\)?|\d{0,2}\.?\d{0,2}(cm|em|ex|in|mm|pc|pt|px|%|,|\))?)$/
-                  goodval = false
-                end
-              end
-              if goodval 
-                clean <<  prop + ': ' + val + ';'
-              end
-            elsif ALLOWED_SVG_PROPERTIES.include?(prop.downcase)
-               clean <<  prop + ': ' + val + ';'
+    module REXML2
+      class TreeWalker < HTML5::TreeWalkers::NonRecursiveTreeWalker
+
+        private
+
+        def node_details(node)
+          case node
+          when ::REXML::Document
+            [:DOCUMENT]
+          when ::REXML::Element
+            if !node.name
+              [:DOCUMENT_FRAGMENT]
+            else
+              [:ELEMENT, node.name,
+                node.attributes.map {|name,value| [name,value.to_utf8]},
+                node.has_elements? || node.has_text?]
             end
+          when ::REXML::Text
+            [:TEXT, node.value.to_utf8]
+          when ::REXML::Comment
+            [:COMMENT, node.string]
+          when ::REXML::DocType
+            [:DOCTYPE, node.name, node.public, node.system]
+          when ::REXML::XMLDecl
+            [nil]
+          else
+            [:UNKNOWN, node.class.inspect]
           end
+        end
 
-          style = clean.join(' ')
+        def first_child(node)
+          node.children.first
+        end
+
+        def next_sibling(node)
+          node.next_sibling
+        end
+
+        def parent(node)
+          node.parent
+        end
       end
-end      
+    end
+  end
+end

attic/test/unit/sanitize_test.rb

@@ -1,187 +1,189 @@
 #!/usr/bin/env ruby
 
-require File.expand_path(File.join(File.dirname(__FILE__), '/../test_helper'))
+require File.expand_path(File.dirname(__FILE__) + '/../test_helper')
 require 'sanitize'
+require 'json'
+
 
 class SanitizeTest < Test::Unit::TestCase
+
   include Sanitize
 
   def setup
 
   end
 
-  Sanitize::ALLOWED_ELEMENTS.each do |tag_name|
+  def do_sanitize_xhtml stream
+    safe_sanitize_xhtml(stream)
+  end
+
+  def check_sanitization(input, htmloutput, xhtmloutput, rexmloutput)
+    assert_equal htmloutput, do_sanitize_xhtml(input)
+  end
+  
+  def rexml_doc(string)
+    REXML::Document.new(
+      "<div xmlns='http://www.w3.org/1999/xhtml'>#{string}</div>")
+  end
+  
+  def my_rex(string)
+    sanitize_rexml(rexml_doc(string.to_utf8)).gsub(/\A<div xmlns="http:\/\/www.w3.org\/1999\/xhtml">(.*)<\/div>\Z/m, '\1')
+  end
+
+  def test_sanitize_named_entities
+    input = '<p>Greek &phis; &phi;, double-struck &Aopf;, numeric &#x1D538; &#8279;, uppercase &TRADE; &LT;</p>'
+    output = "<p>Greek \317\225 \317\206, double-struck \360\235\224\270, numeric \360\235\224\270 \342\201\227, uppercase \342\204\242 &lt;</p>"
+    output2 = "<p>Greek \317\225 \317\206, double-struck \360\235\224\270, numeric &#x1D538; &#8279;, uppercase \342\204\242 &lt;</p>"
+    assert_equal(output, sanitize_xhtml(input))
+    assert_equal(output, sanitize_html(input))
+    assert_equal(output, my_rex(input))
+    assert_equal(output2, input.to_utf8)
+  end
+  
+  def test_sanitize_malformed_utf8
+    input = "<p>\357elephant &AMP; \302ivory</p>"
+    output = "<p>\357\277\275elephant &amp; \357\277\275ivory</p>"
+    check_sanitization(input, output, output, output)
+  end    
+
+  Sanitizer::ALLOWED_ELEMENTS.each do |tag_name|
     define_method "test_should_allow_#{tag_name}_tag" do
-      assert_equal "<#{tag_name} title=\"1\">foo &lt;bad>bar&lt;/bad> baz</#{tag_name}>",
-        sanitize_html("<#{tag_name} title='1'>foo <bad>bar</bad> baz</#{tag_name}>")
+      input       = "<#{tag_name} title='1'>foo <bad>bar</bad> baz</#{tag_name}>"
+      htmloutput  = "<#{tag_name.downcase} title='1'>foo &lt;bad&gt;bar&lt;/bad&gt; baz</#{tag_name.downcase}>"
+      xhtmloutput = "<#{tag_name} title='1'>foo &lt;bad&gt;bar&lt;/bad&gt; baz</#{tag_name}>"
+      rexmloutput = xhtmloutput
+      
+      if %w[caption colgroup optgroup option tbody td tfoot th thead tr].include?(tag_name)
+        htmloutput = "foo &lt;bad&gt;bar&lt;/bad&gt; baz"
+        xhtmloutput = htmloutput
+      elsif tag_name == 'col'
+        htmloutput = "foo &lt;bad&gt;bar&lt;/bad&gt; baz"
+        xhtmloutput = htmloutput
+        rexmloutput = "<col title='1' />"
+      elsif tag_name == 'table'
+        htmloutput = "foo &lt;bad&gt;bar&lt;/bad&gt;baz<table title='1'> </table>"
+        xhtmloutput = htmloutput
+      elsif tag_name == 'image'
+        htmloutput = "<img title='1'/>foo &lt;bad&gt;bar&lt;/bad&gt; baz"
+        xhtmloutput = htmloutput
+        rexmloutput = "<image title='1'>foo &lt;bad&gt;bar&lt;/bad&gt; baz</image>"
+      elsif VOID_ELEMENTS.include?(tag_name)
+        htmloutput = "<#{tag_name} title='1'/>foo &lt;bad&gt;bar&lt;/bad&gt; baz"
+        xhtmloutput = htmloutput
+        htmloutput += '<br/>' if tag_name == 'br'
+        rexmloutput =  "<#{tag_name} title='1' />"
+      end
+      check_sanitization(input, xhtmloutput, xhtmloutput, rexmloutput)
     end
   end
 
-  Sanitize::ALLOWED_ELEMENTS.each do |tag_name|
+  Sanitizer::ALLOWED_ELEMENTS.each do |tag_name|
     define_method "test_should_forbid_#{tag_name.upcase}_tag" do
-      assert_equal "&lt;#{tag_name.upcase} title=\"1\">foo &lt;bad>bar&lt;/bad> baz&lt;/#{tag_name.upcase}>",
-        sanitize_html("<#{tag_name.upcase} title='1'>foo <bad>bar</bad> baz</#{tag_name.upcase}>")
+      input = "<#{tag_name.upcase} title='1'>foo <bad>bar</bad> baz</#{tag_name.upcase}>"
+      output = "&lt;#{tag_name.upcase} title=\"1\"&gt;foo &lt;bad&gt;bar&lt;/bad&gt; baz&lt;/#{tag_name.upcase}&gt;"
+      xhtmloutput = "&lt;#{tag_name.upcase} title='1'&gt;foo &lt;bad&gt;bar&lt;/bad&gt; baz&lt;/#{tag_name.upcase}&gt;"
+      check_sanitization(input, output, xhtmloutput, output)
     end
   end
 
-  Sanitize::ALLOWED_ATTRIBUTES.each do |attribute_name|
-    if attribute_name != 'style'
-      define_method "test_should_allow_#{attribute_name}_attribute" do
-        assert_equal "<p #{attribute_name}=\"foo\">foo &lt;bad>bar&lt;/bad> baz</p>",
-          sanitize_html("<p #{attribute_name}='foo'>foo <bad>bar</bad> baz</p>")
+  Sanitizer::ALLOWED_ATTRIBUTES.each do |attribute_name|
+    next if attribute_name == 'style' || attribute_name.include?(':')
+    define_method "test_should_allow_#{attribute_name}_attribute" do
+      input = "<p #{attribute_name}='foo'>foo <bad>bar</bad> baz</p>"
+      output = "<p #{attribute_name}='foo'>foo &lt;bad&gt;bar&lt;/bad&gt; baz</p>"
+      htmloutput = "<p #{attribute_name.downcase}='foo'>foo &lt;bad&gt;bar&lt;/bad&gt; baz</p>"
+      check_sanitization(input, output, output, output)
+    end
+  end
+
+  Sanitizer::ALLOWED_ATTRIBUTES.each do |attribute_name|
+    define_method "test_should_forbid_#{attribute_name.upcase}_attribute" do
+      input = "<p #{attribute_name.upcase}='display: none;'>foo <bad>bar</bad> baz</p>"
+      output =  "<p>foo &lt;bad&gt;bar&lt;/bad&gt; baz</p>"
+      check_sanitization(input, output, output, output)
+    end
+  end
+
+  Sanitizer::ALLOWED_PROTOCOLS.each do |protocol|
+    define_method "test_should_allow_#{protocol}_uris" do
+      input = %(<a href="#{protocol}">foo</a>)
+      output = "<a href='#{protocol}'>foo</a>"
+      check_sanitization(input, output, output, output)
+    end
+  end
+
+  Sanitizer::ALLOWED_PROTOCOLS.each do |protocol|
+    define_method "test_should_allow_uppercase_#{protocol}_uris" do
+      input = %(<a href="#{protocol.upcase}">foo</a>)
+      output = "<a href='#{protocol.upcase}'>foo</a>"
+      check_sanitization(input, output, output, output)
+    end
+  end
+
+  Sanitizer::SVG_ALLOW_LOCAL_HREF.each do |tag_name|
+    next unless Sanitizer::ALLOWED_ELEMENTS.include?(tag_name)
+    define_method "test_#{tag_name}_should_allow_local_href_with_ns_decl" do
+      input = %(<#{tag_name} xlink:href="#foo" xmlns:xlink='http://www.w3.org/1999/xlink'/>)
+      output = "<#{tag_name.downcase} xlink:href='#foo' xmlns:xlink='http://www.w3.org/1999/xlink'/>"
+      xhtmloutput = "<#{tag_name} xlink:href='#foo' xmlns:xlink='http://www.w3.org/1999/xlink'/>"
+      check_sanitization(input, xhtmloutput, xhtmloutput, xhtmloutput)
+    end
+
+    define_method "test_#{tag_name}_should_allow_local_href_with_newline_and_ns_decl" do
+      input = %(<#{tag_name} xlink:href="\n#foo" xmlns:xlink='http://www.w3.org/1999/xlink'/>)
+      output = "<#{tag_name.downcase} xlink:href='\n#foo' xmlns:xlink='http://www.w3.org/1999/xlink'/>"
+      xhtmloutput = "<#{tag_name} xlink:href='\n#foo' xmlns:xlink='http://www.w3.org/1999/xlink'/>"
+      check_sanitization(input, xhtmloutput, xhtmloutput, xhtmloutput)
+    end
+
+    define_method "test_#{tag_name}_should_forbid_local_href_without_ns_decl" do
+      input = %(<#{tag_name} xlink:href="#foo"/>)
+      output = "&lt;#{tag_name.downcase} xlink:href='#foo'/>"
+      xhtmloutput = "&lt;#{tag_name} xlink:href=&#39;#foo&#39;&gt;&lt;/#{tag_name}&gt;"
+      check_sanitization(input, xhtmloutput, xhtmloutput, xhtmloutput)
+    end
+
+    define_method "test_#{tag_name}_should_forbid_local_href_with_newline_without_ns_decl" do
+      input = %(<#{tag_name} xlink:href="\n#foo"/>)
+      output = "&lt;#{tag_name.downcase} xlink:href='\n#foo'/>"
+      xhtmloutput = "&lt;#{tag_name} xlink:href=&#39;\n#foo&#39;&gt;&lt;/#{tag_name}&gt;"
+      check_sanitization(input, xhtmloutput, xhtmloutput, xhtmloutput)
+    end
+
+    define_method "test_#{tag_name}_should_forbid_nonlocal_href_with_ns_decl" do
+      input = %(<#{tag_name} xlink:href="http://bad.com/foo" xmlns:xlink='http://www.w3.org/1999/xlink'/>)
+      output = "<#{tag_name.downcase} xmlns:xlink='http://www.w3.org/1999/xlink'/>"
+      xhtmloutput = "<#{tag_name} xmlns:xlink='http://www.w3.org/1999/xlink'/>"
+      check_sanitization(input, xhtmloutput, xhtmloutput, xhtmloutput)
+    end
+
+    define_method "test_#{tag_name}_should_forbid_nonlocal_href_with_newline_and_ns_decl" do
+      input = %(<#{tag_name} xlink:href="\nhttp://bad.com/foo" xmlns:xlink='http://www.w3.org/1999/xlink'/>)
+      output = "<#{tag_name.downcase} xmlns:xlink='http://www.w3.org/1999/xlink'/>"
+      xhtmloutput = "<#{tag_name} xmlns:xlink='http://www.w3.org/1999/xlink'/>"
+      check_sanitization(input, xhtmloutput, xhtmloutput, xhtmloutput)
+    end
+  end
+
+  def test_should_handle_astral_plane_characters
+    input = "<p>&#x1d4b5; &#x1d538;</p>"
+    output = "<p>\360\235\222\265 \360\235\224\270</p>"
+    check_sanitization(input, output, output, output)
+
+    input = "<p><tspan>\360\235\224\270</tspan> a</p>"
+    output = "<p><tspan>\360\235\224\270</tspan> a</p>"
+    check_sanitization(input, output, output, output)
+  end
+  
+    JSON::parse(open(File.expand_path(File.join(File.dirname(__FILE__), '/../sanitizer.dat'))).read).each do |test|
+      define_method "test_#{test['name']}" do
+        check_sanitization(
+          test['input'],
+          test['output'],
+          test['xhtml'] || test['output'],
+          test['rexml'] || test['output']
+        )
       end
     end
-  end
-
-  Sanitize::ALLOWED_ATTRIBUTES.each do |attribute_name|
-    define_method "test_should_forbid_#{attribute_name.upcase}_attribute" do
-      assert_equal "<p>foo &lt;bad>bar&lt;/bad> baz</p>",
-        sanitize_html("<p #{attribute_name.upcase}='display: none;'>foo <bad>bar</bad> baz</p>")
-    end
-  end
-
-  Sanitize::ALLOWED_PROTOCOLS.each do |protocol|
-    define_method "test_should_allow_#{protocol}_uris" do
-      assert_equal "<a href=\"#{protocol}\">foo</a>",
-        sanitize_html(%(<a href="#{protocol}">foo</a>))
-    end
-  end
-
-  Sanitize::ALLOWED_PROTOCOLS.each do |protocol|
-    define_method "test_should_allow_uppercase_#{protocol}_uris" do
-      assert_equal "<a href=\"#{protocol.upcase}\">foo</a>",
-        sanitize_html(%(<a href="#{protocol.upcase}">foo</a>))
-    end
-  end
-
-  def test_should_allow_anchors
-    assert_equal "<a href=\"foo\">&lt;script>baz&lt;/script></a>",
-     sanitize_html("<a href='foo' onclick='bar'><script>baz</script></a>")
-  end
-
-  # RFC 3986, sec 4.2
-  def test_allow_colons_in_path_component
-    assert_equal "<a href=\"./this:that\">foo</a>",
-      sanitize_html("<a href=\"./this:that\">foo</a>")
-  end
-
-  %w(src width height alt).each do |img_attr|
-    define_method "test_should_allow_image_#{img_attr}_attribute" do
-      assert_equal "<img #{img_attr}=\"foo\" />",
-        sanitize_html("<img #{img_attr}='foo' onclick='bar' />")
-    end
-  end
-
-  def test_should_handle_non_html
-    assert_equal 'abc',  sanitize_html("abc")
-  end
-
-  def test_should_handle_blank_text
-    assert_equal '', sanitize_html('')
-  end
-
-  [%w(img src), %w(a href)].each do |(tag, attr)|
-    define_method "test_should_strip_#{attr}_attribute_in_#{tag}_with_bad_protocols" do
-      assert_equal %(<#{tag} title="1">boo</#{tag}>), sanitize_html(%(<#{tag} #{attr}="javascript:XSS" title="1">boo</#{tag}>))
-    end
-  end
-
-  [%w(img src), %w(a href)].each do |(tag, attr)|
-    define_method "test_should_strip_#{attr}_attribute_in_#{tag}_with_bad_protocols_and_whitespace" do
-      assert_equal %(<#{tag} title="1">boo</#{tag}>), sanitize_html(%(<#{tag} #{attr}=" javascript:XSS" title="1">boo</#{tag}>))
-    end
-  end
-
-  [%(<img src="javascript:alert('XSS');" />), 
-   %(<img src=javascript:alert('XSS') />), 
-   %(<img src="JaVaScRiPt:alert('XSS')" />), 
-   %(<img src='javascript:alert(&quot;XSS&quot;)' />),
-   %(<img src='javascript:alert(String.fromCharCode(88,83,83))' />),
-   %(<img src='&#106;&#97;&#118;&#97;&#115;&#99;&#114;&#105;&#112;&#116;&#58;&#97;&#108;&#101;&#114;&#116;&#40;&#39;&#88;&#83;&#83;&#39;&#41;' />),
-   %(<img src='&#0000106;&#0000097;&#0000118;&#0000097;&#0000115;&#0000099;&#0000114;&#0000105;&#0000112;&#0000116;&#0000058;&#0000097;&#0000108;&#0000101;&#0000114;&#0000116;&#0000040;&#0000039;&#0000088;&#0000083;&#0000083;&#0000039;&#0000041' />),
-   %(<img src='&#x6A;&#x61;&#x76;&#x61;&#x73;&#x63;&#x72;&#x69;&#x70;&#x74;&#x3A;&#x61;&#x6C;&#x65;&#x72;&#x74;&#x28;&#x27;&#x58;&#x53;&#x53;&#x27;&#x29' />),
-   %(<img src="jav\tascript:alert('XSS');" />),
-   %(<img src="jav&#x09;ascript:alert('XSS');" />),
-   %(<img src="jav&#x0A;ascript:alert('XSS');" />),
-   %(<img src="jav&#x0D;ascript:alert('XSS');" />),
-   %(<img src=" &#14;  javascript:alert('XSS');" />),
-   %(<img src="&#x20;javascript:alert('XSS');" />),
-   %(<img src="&#xA0;javascript:alert('XSS');" />)].each_with_index do |img_hack, i|
-    define_method "test_should_not_fall_for_xss_image_hack_#{i}" do
-      assert_equal "<img />", sanitize_html(img_hack)
-    end
-  end
-
-  def test_should_sanitize_tag_broken_up_by_null
-    assert_equal "&lt;scr>alert(\"XSS\")&lt;/scr>", sanitize_html(%(<scr\0ipt>alert(\"XSS\")</scr\0ipt>))
-  end
-  
-  def test_should_sanitize_invalid_script_tag
-    assert_equal "&lt;script />&lt;/script>", sanitize_html(%(<script/XSS SRC="http://ha.ckers.org/xss.js"></script>))
-  end
-  
-  def test_should_sanitize_script_tag_with_multiple_open_brackets
-    assert_equal "&lt;&lt;script>alert(\"XSS\");//&lt;&lt;/script>", sanitize_html(%(<<script>alert("XSS");//<</script>))
-    assert_equal %(&lt;iframe src="http:" />&lt;), sanitize_html(%(<iframe src=http://ha.ckers.org/scriptlet.html\n<))
-  end
-  
-  def test_should_sanitize_unclosed_script
-    assert_equal "&lt;script src=\"http:\" /><b>", sanitize_html(%(<script src=http://ha.ckers.org/xss.js?<b>))
-  end
-  
-  def test_should_sanitize_half_open_scripts
-    assert_equal  "<img>", sanitize_html(%(<img src="javascript:alert('XSS')"))
-  end
-  
-  def test_should_not_fall_for_ridiculous_hack
-    img_hack = %(<img\nsrc\n=\n"\nj\na\nv\na\ns\nc\nr\ni\np\nt\n:\na\nl\ne\nr\nt\n(\n'\nX\nS\nS\n'\n)\n"\n />)
-    assert_equal "<img />", sanitize_html(img_hack)
-  end
-
-  def test_platypus
-    assert_equal %(<a href=\"http://www.ragingplatypus.com/\" style=\"display: block; width: 100%; height: 100%; background-color: black; background-image: ; background-x: center; background-y: center;\">never trust your upstream platypus</a>),
-       sanitize_html(%(<a href="http://www.ragingplatypus.com/" style="display:block; position:absolute; left:0; top:0; width:100%; height:100%; z-index:1; background-color:black; background-image:url(http://www.ragingplatypus.com/i/cam-full.jpg); background-x:center; background-y:center; background-repeat:repeat;">never trust your upstream platypus</a>))
-  end
-
-  def test_xul
-    assert_equal %(<p style="">fubar</p>),
-     sanitize_html(%(<p style="-moz-binding:url('http://ha.ckers.org/xssmoz.xml#xss')">fubar</p>))
-  end
-
-  def test_input_image
-    assert_equal %(<input type="image" />),
-      sanitize_html(%(<input type="image" src="javascript:alert('XSS');" />))
-  end
-
-  def test_non_alpha_non_digit
-    assert_equal "&lt;script />&lt;/script>",
-      sanitize_html(%(<script/XSS src="http://ha.ckers.org/xss.js"></script>))
-    assert_equal "<a>foo</a>",
-      sanitize_html('<a onclick!#$%&()*~+-_.,:;?@[/|\]^`=alert("XSS")>foo</a>')
-    assert_equal "<img />",
-      sanitize_html('<img/src="http://ha.ckers.org/xss.js"/>')
-  end
-
-  def test_img_dynsrc_lowsrc
-     assert_equal "<img />",
-       sanitize_html(%(<img dynsrc="javascript:alert('XSS')" />))
-     assert_equal "<img />",
-       sanitize_html(%(<img lowsrc="javascript:alert('XSS')" />))
-  end
-
-  def test_div_background_image_unicode_encoded
-    assert_equal '<div style="">foo</div>',
-      sanitize_html(%(<div style="background-image:\0075\0072\006C\0028'\006a\0061\0076\0061\0073\0063\0072\0069\0070\0074\003a\0061\006c\0065\0072\0074\0028.1027\0058.1053\0053\0027\0029'\0029">foo</div>))
-  end
-
-  def test_div_expression
-    assert_equal '<div style="">foo</div>',
-      sanitize_html(%(<div style="width: expression(alert('XSS'));">foo</div>))
-  end
-
-  def test_img_vbscript
-     assert_equal '<img />',
-       sanitize_html(%(<img src='vbscript:msgbox("XSS")' />))
-  end
 
 end

attic/vendor/plugins/HTML5lib/lib/html5/sanitizer.rb

@@ -1,5 +1,6 @@
 require 'cgi'
 require 'html5/tokenizer'
+require 'set'
 
 module HTML5
 
@@ -19,25 +20,25 @@ module HTML5
 
   module HTMLSanitizeModule
 
-    ACCEPTABLE_ELEMENTS = %w[a abbr acronym address area audio b big blockquote br
+    ACCEPTABLE_ELEMENTS = Set.new %w[a abbr acronym address area audio b big blockquote br
       button caption center cite code col colgroup dd del dfn dir div dl dt
       em fieldset font form h1 h2 h3 h4 h5 h6 hr i img input ins kbd label
       legend li map menu ol optgroup option p pre q s samp select small span
       strike strong sub sup table tbody td textarea tfoot th thead tr tt u
       ul var video]
 
-    MATHML_ELEMENTS = %w[annotation annotation-xml maction math merror mfrac
+    MATHML_ELEMENTS = Set.new %w[annotation annotation-xml maction math merror mfrac
       mfenced mi mmultiscripts mn mo mover mpadded mphantom mprescripts mroot mrow
       mspace msqrt mstyle msub msubsup msup mtable mtd mtext mtr munder
       munderover none semantics]
 
-    SVG_ELEMENTS = %w[a animate animateColor animateMotion animateTransform
+    SVG_ELEMENTS = Set.new %w[a animate animateColor animateMotion animateTransform
       circle clipPath defs desc ellipse font-face font-face-name font-face-src
       foreignObject g glyph hkern linearGradient line marker metadata
       missing-glyph mpath path polygon polyline radialGradient rect set
       stop svg switch text title tspan use]
 
-    ACCEPTABLE_ATTRIBUTES = %w[abbr accept accept-charset accesskey action
+    ACCEPTABLE_ATTRIBUTES = Set.new %w[abbr accept accept-charset accesskey action
       align alt axis border cellpadding cellspacing char charoff charset
       checked cite class clear cols colspan color compact controls coords datetime
       dir disabled enctype for frame headers height href hreflang hspace id
@@ -46,7 +47,7 @@ module HTML5
       selected shape size span src start style summary tabindex target title
       type usemap valign value vspace width xml:lang]
 
-    MATHML_ATTRIBUTES = %w[actiontype align close columnalign columnalign
+    MATHML_ATTRIBUTES = Set.new %w[actiontype align close columnalign columnalign
       columnalign columnlines columnspacing columnspan depth display
       displaystyle encoding equalcolumns equalrows fence fontstyle fontweight
       frame height linethickness lspace mathbackground mathcolor mathvariant
@@ -54,7 +55,7 @@ module HTML5
       rowspacing rowspan rspace scriptlevel selection separator separators
       stretchy width width xlink:href xlink:show xlink:type xmlns xmlns:xlink]
 
-    SVG_ATTRIBUTES = %w[accent-height accumulate additive alphabetic
+    SVG_ATTRIBUTES = Set.new %w[accent-height accumulate additive alphabetic
        arabic-form ascent attributeName attributeType baseProfile bbox begin
        by calcMode cap-height class clip-path clip-rule color color-rendering
        content cx cy d dx dy descent display dur end fill fill-opacity fill-rule
@@ -76,16 +77,16 @@ module HTML5
        xlink:show xlink:title xlink:type xml:base xml:lang xml:space xmlns
        xmlns:xlink y y1 y2 zoomAndPan]
 
-    ATTR_VAL_IS_URI = %w[href src cite action longdesc xlink:href xml:base]
+    ATTR_VAL_IS_URI = Set.new %w[href src cite action longdesc xlink:href xml:base]
 
-    SVG_ATTR_VAL_ALLOWS_REF = %w[clip-path color-profile cursor fill
+    SVG_ATTR_VAL_ALLOWS_REF = Set.new %w[clip-path color-profile cursor fill
       filter marker marker-start marker-mid marker-end mask stroke]
 
-    SVG_ALLOW_LOCAL_HREF = %w[altGlyph animate animateColor animateMotion
+    SVG_ALLOW_LOCAL_HREF = Set.new %w[altGlyph animate animateColor animateMotion
       animateTransform cursor feImage filter linearGradient pattern
       radialGradient textpath tref set use]
 
-    ACCEPTABLE_CSS_PROPERTIES = %w[azimuth background-color
+    ACCEPTABLE_CSS_PROPERTIES = Set.new %w[azimuth background-color
       border-bottom-color border-collapse border-color border-left-color
       border-right-color border-top-color clear color cursor direction
       display elevation float font font-family font-size font-style
@@ -95,16 +96,16 @@ module HTML5
       text-align text-decoration text-indent unicode-bidi vertical-align
       voice-family volume white-space width]
 
-    ACCEPTABLE_CSS_KEYWORDS = %w[auto aqua black block blue bold both bottom
+    ACCEPTABLE_CSS_KEYWORDS = Set.new %w[auto aqua black block blue bold both bottom
       brown center collapse dashed dotted fuchsia gray green !important
       italic left lime maroon medium none navy normal nowrap olive pointer
       purple red right solid silver teal top transparent underline white
       yellow]
 
-    ACCEPTABLE_SVG_PROPERTIES = %w[fill fill-opacity fill-rule stroke
+    ACCEPTABLE_SVG_PROPERTIES = Set.new %w[fill fill-opacity fill-rule stroke
       stroke-width stroke-linecap stroke-linejoin stroke-opacity]
 
-    ACCEPTABLE_PROTOCOLS = %w[ed2k ftp http https irc mailto news gopher nntp
+    ACCEPTABLE_PROTOCOLS = Set.new %w[ed2k ftp http https irc mailto news gopher nntp
       telnet webcal xmpp callto feed urn aim rsync tag ssh sftp rtsp afs]
 
     # subclasses may define their own versions of these constants