(major change in big-config mode) split the compiled config file

Fedora's config has over 11,000 repositories and the compiled config
file is over 20 MB in size.  Although negligible on a server class
machine, on my laptop just parsing this file takes a good 2.5 seconds.

Even if you use GL_ALL_READ_ALL (see a couple of commits before this
one) to remove the overhead for 'read's, that's still a pretty big
overhead for writes.  And GL_ALL_READ_ALL is not really a solution for
most people anyway.

With this commit, using GL_BIG_CONFIG adds another optimisation; see
doc/big-config.mkd for details (look for the word "split config" to find
the section that talks about it).

----

Implementation notes:

  - the check for GL_NO_CREATE_REPOS has moved *into* the loop (which it
    completely bypassed earlier) so that write_1_compiled_conf can be
    called on each item
This commit is contained in:
Sitaram Chamarty 2011-01-01 15:14:54 +05:30
parent 7fc1e9459f
commit 10a30c961d
9 changed files with 326 additions and 161 deletions

View file

@ -38,6 +38,7 @@ our ($R_COMMANDS, $W_COMMANDS, $REPONAME_PATT, $REPOPATT_PATT, $ADC_CMD_ARGS_PAT
our %repos;
our %groups;
our %git_configs;
our %split_conf;;
# the common setup module is in the same directory as this running program is
my $bindir = $0;