| 1 |
$Id: README.txt,v 1.10 2007/12/07 23:38:07 allisterbeharry Exp $
|
| 2 |
|
| 3 |
Drupal Automated Staging Toolkit README
|
| 4 |
---------------------------------------
|
| 5 |
|
| 6 |
Current Version
|
| 7 |
---------------
|
| 8 |
5.x-2.1
|
| 9 |
|
| 10 |
|
| 11 |
Release Notes
|
| 12 |
-------------
|
| 13 |
This is the fourth public release of DAST-the second after 5.x-2.0-soc07final.
|
| 14 |
It contains major improvements to the formatting and content of
|
| 15 |
documentation files like this, two new quickstart HowTos, and a new
|
| 16 |
command-line option - p which simplifies the process of running build
|
| 17 |
from the command-line. Just set -p to a directory "myproject" and DAST will
|
| 18 |
load from myproject.xml and myproject-base.properties and myproject-properties
|
| 19 |
from the my project directory. An important change to the way DAST works is that
|
| 20 |
property files are now scanned for relative to the project directory, instead
|
| 21 |
of relative to DAST_HOME. See CHANGELOG.txt for a complete rundown of bug-fixes
|
| 22 |
and new features.
|
| 23 |
|
| 24 |
A PDF snapshot of the current documentation has been included in the docs
|
| 25 |
folder. The core of the toolkit for fetching, configuring, installiing, and
|
| 26 |
patching a Drupal site has been completed. The four core build-files slated for
|
| 27 |
the 2.0 release are complete: site-fetch, site-configure, site-patch, and
|
| 28 |
site-install; and ready to be used by DAST end-users. All the required Phing
|
| 29 |
tasks and types used in these core build files are now complete and ready for
|
| 30 |
use by DAST developers. The new Phing types for reading and writing directives
|
| 31 |
and sections to Apache .conf, PHP ini and MySQL .ini configuration files are
|
| 32 |
complete. See the documentation home page for an understanding of the design
|
| 33 |
and implementation of this toolkit:
|
| 34 |
http://docs.google.com/View?docid=dcgv33zb_25d26w5n.
|
| 35 |
|
| 36 |
|
| 37 |
About
|
| 38 |
-----
|
| 39 |
DAST is a set of build projects, tasks and other extensions created on the
|
| 40 |
Phing(http://www.phing.info) framework for automating a wide variety of Drupal
|
| 41 |
testing and staging scenarios. The goal of DAST is to provide a formal,
|
| 42 |
repeatable, reusable process using an existing build framework for 'building' a
|
| 43 |
Drupal site using a formal build tool and build language. DAST is targeted at
|
| 44 |
Drupal testers and developers who must repeatably setup a Drupal site using
|
| 45 |
different configurations and code revisions for profile and module development
|
| 46 |
and testing, patch testing,and configuration stability testing and benchmarking.
|
| 47 |
But DAST can also be used by Drupal end-users - for example an installation
|
| 48 |
profile developer can bundle a DAST script with her profile so that the user of
|
| 49 |
the profile can have her Drupal site setup and bootstrapped with a minimum of
|
| 50 |
effort - this would be a lot easier than the current manual and error-prone
|
| 51 |
process of bringing up a Drupal site from scratch before running the profile.
|
| 52 |
|
| 53 |
If you're tired of the download/untar/copy/drop/create database routine and you
|
| 54 |
want to automate testing patches and modules then you're in the right place. If
|
| 55 |
you're looking for a tool for Drupal build management and to do stuff like
|
| 56 |
automated unit testing and and anything that can be done to make
|
| 57 |
Drupal's devs life easier and improve code quality, you're also in the right
|
| 58 |
place. See the documentation page
|
| 59 |
at http://docs.google.com/View?docid=dcgv33zb_25d26w5n for a complete guide to
|
| 60 |
the motivation, concepts, implementation, and features of the project.
|
| 61 |
|
| 62 |
|
| 63 |
Dependencies
|
| 64 |
------------
|
| 65 |
0. PHP 5.2.x - DAST is a CLI app for *nix/Windows that requires PHP 5.2.x or
|
| 66 |
higher , it does not run on anything lower than PHP 5.2.0. DAST also requires
|
| 67 |
PHP-PEAR to be installed for the current PHP version.
|
| 68 |
|
| 69 |
1. For the forseeable future DAST will be released with all PHP/PEAR
|
| 70 |
library dependencies bundled in a single package. The dependencies package can
|
| 71 |
be obtained from http://www.abeharry.info/stuff/dast-deps.tar.gz or
|
| 72 |
http://www.abeharry.info/stuff/dast-deps.zip.
|
| 73 |
The current dependencies are:
|
| 74 |
-Phing 2.3.0dev Phing (www.phing.info) is the excellent PHPized Ant build
|
| 75 |
framework, capable of deploying just about any PHP application using concepts
|
| 76 |
and syntax identical to Ant.
|
| 77 |
-Creole 1.1.0 Creole (creole.phpdb.org) is an excellent OO database
|
| 78 |
abstraction library for PHP by the same folks who developed Phing.
|
| 79 |
-Jargon Jargon (creole.phpdb.org/trac/wiki/Documentation/JargonGuide)
|
| 80 |
is a high-level library built on Creole for performing common db data
|
| 81 |
operations.
|
| 82 |
-SimpleTest SimpleTest (http://simpletest.sourceforge.net) is a
|
| 83 |
popular testing framework for PHP applications. Much of the existing unit
|
| 84 |
testing in Drupal including the simpletestauto module uses SimpleTest.
|
| 85 |
-PHPUnit Another unit-testing framework that can be used - the
|
| 86 |
SimpleTest functionality in Phing has some dependencies on this.
|
| 87 |
-PEAR::Config An excellent PEAR library by Bertrand Mansion
|
| 88 |
<bmansion@mamasam.com> for reading and writing to
|
| 89 |
different config file formats including Apache con fand PHP .ini formats.
|
| 90 |
-PHP-PEAR The version of PHP-PEAR which ships with PHP 5 is
|
| 91 |
required; some bundled packages have dependencies on the base PEAR packages.
|
| 92 |
|
| 93 |
Since DAST can and does modify the code in these libraries, in the interests
|
| 94 |
of sanity it bundles its own private
|
| 95 |
versions of these libraries (except base PEAR) in its own dependencies
|
| 96 |
package. It does not touch any other versions of these libraries you have
|
| 97 |
installed with PEAR or otherwise. During startup the shell wrapper explictly
|
| 98 |
puts DAST's own libraries ahead of any others on the PHP include_path. However,
|
| 99 |
since these dependencies are bona fide PEAR packages, you need to have at least
|
| 100 |
the bare-bones PEAR system available.
|
| 101 |
|
| 102 |
2. WinNT > 4 if on Windows, any version of NT4/2000/XP should work. I will test
|
| 103 |
DAST on Vista as soon as I get the opportunity.
|
| 104 |
|
| 105 |
3. External tools - DAST and Phing build scripts may require the presence of
|
| 106 |
external command-line tools normally present on a development environment. DAST
|
| 107 |
currently requires cvs, wget, and patch available on the command line.
|
| 108 |
Windows users can download a version of wget for Windows at
|
| 109 |
http://www.abeharry.info/stuff/dast-ext-tools/wget.exe.
|
| 110 |
|
| 111 |
|
| 112 |
Directory Layout
|
| 113 |
----------------
|
| 114 |
After installation (see INSTALL.txt) your directory structure will look like
|
| 115 |
this:
|
| 116 |
DAST_HOME - wherever you unpack the DAST package on your filesystem
|
| 117 |
|--bin This folder contains all the shell wrappers and executable PHP CLI
|
| 118 |
scripts
|
| 119 |
|--creole Contains the Creole library
|
| 120 |
|--doc User guide and API documentation
|
| 121 |
|--ext External scripts and tools used by build files, like patch scripts
|
| 122 |
|--jargon Contains the Jargon library
|
| 123 |
|--log Contains build file logs
|
| 124 |
|--projects You can use this folder for your build file projects. Contains the
|
| 125 |
default-site, do_testing-site and other quickstart projects.
|
| 126 |
|--phing Contains the Phing framework
|
| 127 |
|--PEAR_Config Contains the PEAR::Config library
|
| 128 |
|--simpletest Contains the SimpleTest library
|
| 129 |
|--PHPUnit Contains the PHPUnit library
|
| 130 |
|--src This folder contains all DAST-specific PHP class files and other source
|
| 131 |
files
|
| 132 |
|--tmp DAST temporary files directory
|
| 133 |
|
| 134 |
|
| 135 |
Installing
|
| 136 |
--------------
|
| 137 |
See INSTALL.txt
|
| 138 |
|
| 139 |
|
| 140 |
Using
|
| 141 |
-----
|
| 142 |
1. Set your properties for the build. Each .xml build file has an associated
|
| 143 |
properties file with the extension ".properties" - e.g. the do_testing-site.xml
|
| 144 |
build file in the projects/do_testing-site sub-directory has the properties
|
| 145 |
which control the build process defined in the file do_testing-site.properties.
|
| 146 |
Properties are simple name/value pairs with comments delimited by #. By
|
| 147 |
convention all required properties are stated in a comment block at the start
|
| 148 |
of the properties file. In the DAST_HOME root are sample properties files for
|
| 149 |
*nix and Windows for each of the 4 core build files. These contain all the
|
| 150 |
properties that are likely to be used in a build project. If you want to use
|
| 151 |
one of these core build files directly make a copy of one of these properties
|
| 152 |
files according to your OS in the DAST root directory and and modify it with
|
| 153 |
your settings to configure the build. But normally you will be working with
|
| 154 |
build projects, located in the projects sub-directory. Build projects have
|
| 155 |
their own properties files and call the core build files which inherit the
|
| 156 |
properties defined in the project's properties file. You specify a properties
|
| 157 |
file for a build file mybuild.xml by naming it mybuild.properties, or by using
|
| 158 |
the -D command-line parameter, see 2 on how to specify properties files on the
|
| 159 |
command line.
|
| 160 |
|
| 161 |
|
| 162 |
2. Run your build script - To run the build file projects/do_testing-site.xml
|
| 163 |
run the folowing command from DAST_HOME:
|
| 164 |
|
| 165 |
./bin/dast -f ./projects/do_testing-site/do_testing-site.xml
|
| 166 |
|
| 167 |
./bin/dast is the shell wrapper which kicks off the PHP CLI script. The
|
| 168 |
f parameter denotes the location of the build file you want to run. You
|
| 169 |
should always prepend a relative path specifier (e.g for current directory -./)
|
| 170 |
to your build file names. By default the properties file in your build file's
|
| 171 |
base dir (in this case ./project/do_testing-site) will be used to load
|
| 172 |
properties from. If you want to use another properties file, specify it using
|
| 173 |
the -D parameter:
|
| 174 |
|
| 175 |
./bin/dast -f ...<myfile>.xml -D"propertiesFile=myprops.properties"
|
| 176 |
|
| 177 |
You can also use the -p parameter which indicates a project name. Projects
|
| 178 |
are simply build files together with their properties files which reside in
|
| 179 |
their own directory under the DAST_HOME/projects directory. If you have a
|
| 180 |
myproject.xml build file and you put it in the directory
|
| 181 |
DAST_HOME/projects/myproject together with myproject.properties and/or
|
| 182 |
myproject-base.properties you can then run:
|
| 183 |
|
| 184 |
./bin/dast -p myproject
|
| 185 |
This is equivalent to
|
| 186 |
./bin/dast -f ./projects/myproject -D"propertiesFile=
|
| 187 |
./projects/myproject/myproject.properties" -D"basePropertiesFile=
|
| 188 |
./projects/myproject/myproject-base.properties"
|
| 189 |
|
| 190 |
The -D parameter is actually a general parameter that denotes properties you
|
| 191 |
wish to pass in to the build process. The 'propertiesFile' property denotes the
|
| 192 |
location of the .properties file to load all the remaining properties to
|
| 193 |
use in the build process. Any properties defined in the properties can be
|
| 194 |
overriden using -D"<property>=<value>". So suppose you wanted to run the
|
| 195 |
do_testing-site build using mysite1.proerties as the properties file,
|
| 196 |
the drupal.dir as /var/www/site1 and drupal.core.method = tar you could do:
|
| 197 |
|
| 198 |
./bin/dast -f ./projects/do_testing-site/do_testing-site.xml
|
| 199 |
-D"propertiesFile=mysite1.properties" -D"drupal.dir=/var/www/site1"
|
| 200 |
-D"drupal.core.method=tar"
|
| 201 |
|
| 202 |
Properties files may be split into a "base" properties file and a "user"
|
| 203 |
properties file. You define a base properties
|
| 204 |
file for your-build.xml by either passing it as the command-line property
|
| 205 |
basePropetiesFile(D"basePropertiesFile=..") or naming it your-build-
|
| 206 |
base.properties. You may then create a normal properties file, called here a
|
| 207 |
user properties file as described above. What will happen is
|
| 208 |
that when the build runs it will load both properties files. Properties not
|
| 209 |
defined in the user properties file will be inherited from the base properties
|
| 210 |
files. Properties defined in the user properties file will override those
|
| 211 |
defined in the base properties files. In this way you may keep a properties file
|
| 212 |
containg common settings for several builds and create smaller more specific
|
| 213 |
properties files to customize each individual build. Note that specifying the -D
|
| 214 |
parameter for a base properties file will prevent the default user properties
|
| 215 |
file from being loaded...i.e for yourbuild.xml if you use
|
| 216 |
-D"basePropertiesFile=<anotheyourbuild-base.properties>" then DAST will
|
| 217 |
not look for yourbuild.properties as the default user properties file.
|
| 218 |
You must explicitly specify your user properties file if not using the default
|
| 219 |
base properties file name.
|
| 220 |
|
| 221 |
You may also specify you want to log all output to a file using the -logfile
|
| 222 |
<logfile> parameter.
|
| 223 |
|
| 224 |
All unqualified file locations and relative paths passed as parameters are
|
| 225 |
assumed relative to the DAST_HOME directory, regardless of what the current
|
| 226 |
directory is when you run the build command. If you want to use build files or
|
| 227 |
property files or other files outside DAST_HOME, use a fully qualified name.
|
| 228 |
|
| 229 |
3. Check the status of the build when the script completes - Phing logs
|
| 230 |
extensive status information to the screen; check the results of each task in
|
| 231 |
the build to see if it completed successfully or if any errors were reported.
|
| 232 |
DAST also logs extensive info to the log dir set in build.log.defaultdir
|
| 233 |
(default /var/log/DAST or E:\var\log) so if you encounter any errors be sure to
|
| 234 |
check these logs.
|
| 235 |
|
| 236 |
4. That's it, you're done. You now have a complete working Drupal site for
|
| 237 |
development or testing or deployment. For the default profile you only have to
|
| 238 |
create the admin account next. Profiles like drupalorg_testing will do this
|
| 239 |
automatically.
|
| 240 |
|
| 241 |
|
| 242 |
|
| 243 |
Command-line Args
|
| 244 |
-----------------
|
| 245 |
-p <project_name> Execute the project_name.xml build file in
|
| 246 |
DAST_HOME/projects/project_name with user properties file
|
| 247 |
project_name.properties and base properties file
|
| 248 |
project_name-base.properties. This is the recommended way to use DAST.
|
| 249 |
-logfile <file> use given file for log
|
| 250 |
-f -buildfile <file> use given buildfile
|
| 251 |
-D<property>=<value> use value for given property
|
| 252 |
|
| 253 |
Phing uses the following additional command-line argumets:
|
| 254 |
-h -help print this message
|
| 255 |
-l -list list available targets in this project
|
| 256 |
-v -version print the version information and exit
|
| 257 |
-q -quiet be extra quiet
|
| 258 |
-verbose be extra verbose
|
| 259 |
-debug print debugging information
|
| 260 |
-logger <classname> the class which is to perform logging
|
| 261 |
-find <file> search for buildfile towards the root of the
|
| 262 |
filesystem and use it
|
| 263 |
|
| 264 |
|
| 265 |
Colour codes
|
| 266 |
------------
|
| 267 |
Under *nix the Phing ANSI logger is used which allows the use of colour-coded
|
| 268 |
text messages. Phing builds always provide detailed logging to the screen as the
|
| 269 |
build progresses; the following is a guide to the different colours used in the
|
| 270 |
screen log text:
|
| 271 |
-Blue OK, informational messages echoed to the screen, also 'verbose' output
|
| 272 |
from task operations
|
| 273 |
-Green OK, informational messages indicating which tasks are executing
|
| 274 |
-Cyan(lightish Blue) OK, informational messages as task operations execute
|
| 275 |
-Magneta(pink) Not so OK, a warning message emitted from a task operation
|
| 276 |
-Red Bad, A task declared the build failed for some reason or an unexpected
|
| 277 |
PHP exception or system error was raised. Check the message details and
|
| 278 |
verify that the relavent properties are set correctly - ensure properties
|
| 279 |
like database Urls and the CVSROOT variables are in a valid format.
|
| 280 |
|
| 281 |
|
| 282 |
Known Issues
|
| 283 |
------------
|
| 284 |
1. Under Windows certain versions of wget (my version at least - MINGWport) may
|
| 285 |
ignore any part of the Url after the ampersand even when escaped.
|
| 286 |
wget "http://localhost/drupal5/dev/dast/index.php?profile=default&locale=en"
|
| 287 |
only fetches http://localhost/drupal5/dev/dast/index.php?profile=default This
|
| 288 |
means that you still need to go through the 'Choose locale' part. This isn't how
|
| 289 |
wget is supposed to behave (if the Url is quoted) so I have to confirm if this
|
| 290 |
is just an eccentricity of the version I have on my system.
|
| 291 |
|
| 292 |
2. Task log files paths can't contain spaces all - if say you have DAST
|
| 293 |
installed in e:\Drupal projects\dast then setting a task log file to point at
|
| 294 |
.\logs\dast-patch-install-wget.log won't work because the path will expand to
|
| 295 |
e:\Drupal projects\dast. This doesn't apply to the Phing log file which you set
|
| 296 |
using the -logfile parameter.
|
| 297 |
|
| 298 |
3. If you use the create-files-dir task, which creates the files directory under
|
| 299 |
drupal.dir and chmods it to 0777, the tasks works as expected but cleaning the
|
| 300 |
site after will fail because the .htaccess Drupal drops in there is
|
| 301 |
readonly and the build file will not be able to delete it. The default-site and
|
| 302 |
do_testing-site projects come with a task call clean-files with which you can
|
| 303 |
sudo run to remove the files directory cleanly before the main build so you
|
| 304 |
could go
|
| 305 |
sudo ./bin/dast -p default-site clean-files
|
| 306 |
./bin/dast -p default-site
|
| 307 |
The next version I will try to figure out a way to delete the .htaccess file
|
| 308 |
(suggestions are welcome.) For many testing and dev scenarios you may not need
|
| 309 |
to use the files directory though.
|
| 310 |
|
| 311 |
Docs
|
| 312 |
----
|
| 313 |
1. The main documentation page is on Google Docs at http://docs.google.com/Doc?id=dcgv33zb_25d26w5n.
|
| 314 |
2. See also the Phing user manual in DAST_HOME/phing/docs/phing_guide and also the Diving into Phing...series of
|
| 315 |
articles in the SoC2007 group on the Drupal website -
|
| 316 |
http://groups.drupal.org/soc-2007 and the DAST wiki page @
|
| 317 |
http://groups.drupal.org/node/3913
|
Parent Directory
| 
Revision Log
| 
Revision Graph