modules/datasync/README.txt

; $Id: README.txt,v 1.1.2.4 2009/07/28 20:41:40 andrewlevine Exp $

description
-------------
The DataSync module was written to schedule and execute jobs in a much more scalable
 and reliable way than with cron.php and hook_cron. It allows you to schedule and run 
module-definable types of jobs on multiple servers in a centralized way. 
It uses database transactions in order to ensure that no data is corrupted while 
running multiple jobs at the same time. For this reason, only MySQL is supported and
you must ensure all tables that must have consistent data are InnoDB tables (Postgres
support is now experimental). This use of transactions allows you to run multiple jobs
(on multiple servers, if you want) touching the same database tables at the same time. 
DataSync keeps track of the status of jobs and what phase they are in, so if a job 
fails it can intelligently decide how to proceed. Almost all of DataSync's default 
behavior is overridable when you create your own job types.

the datasync api
-----------------
DataSync provides an API to schedule, keep track of, and run your own jobs. To implement
the API, you create a "job type" by extending the datasync_job class and registering 
your own class with the hook_datasync_jobs hook. DataSync will then automatically run 
your jobs after they are added with a datasync_job_add() call.

The first thing you have to do to create a job type is create a blank new module. Next,
you need to create a file called $JOB_TYPE.job.inc in that module's folder where $JOB_TYPE
is the name of your new job type. It should extend the datasync_job class so its definition
will be something like this:
    class my_job_type extends datasync_job {

Note: You can also extend another already implemented job type instead but the class hierarchy 
must begin with datasync_job.

In this class you have to implement, at a minimum, the phase_defs method and a method for each
of your phases. The phase_defs method is implemented like this:
    public function phase_defs() {
In this method you just return an array of strings that represent the phases your job has to go
through (order is important). DataSync will simply step through these phases, one by one, as 
fast as it can. So for example, this would be a full phase_defs implementation:
   public function phase_defs() {
     return array('begin', 'read_data', 'save_data', 'exit'); 
   }
Next, you must implement a method for each one of your phases. You can have a minimum of one
phase. But in this example you would have to implement:
  public function begin() {
  public function read_data() {
  public function save_data() {
  public function exit() {
Each of these methods should return TRUE on success and FALSE on anything else.

Next, you need to implement the hook_datasync_job_types hook in the module you have created. So for 
example, if the module we have created is called my_job_type, our definition would look something
like this:
  function my_job_type_datasync_job_types() {
    return array(
      'my_job_type' => array(
        'human_name' => t('Name of My Job Type'),
        'description' => t('Description of my job type'),
        'parent' => 'datasync_job',
      ),
    );
  }
The key of the array you return must be the same of the class you created, the human_name and description
elements are the human readable descriptors of your job type, and parent must be the name of the
class you extended (usually datasync_job).

At this point, assuming you've set up DataSync as specified in INSTALL.txt, when you add a job with:
  datasync_job_add($YOUR_JOB_TYPE);
DataSync should automatically run your job. 

datasync_scheduler
-------------------
The datasync_scheduler module is included in the datasync module package and provides a user interface
to schedule exactly when you want jobs of different types to be created and timed out. It also hooks into
the main datasync module (with the datasync_post_jobs hook) and runs/times out these jobs according to
the schedule you created. Check the README.txt in that module's folder for more information and about
APIs that datasync_scheduler provides.

datasync_cronalt
-------------------
Check the README.txt in this module's subdirectory.


If you have any questions, feel free to contact me (Andrew Levine) through my
drupal.org user contact page:
http://drupal.org/user/49940/contact


extra info on the api:
It may also be helpful for you to know how datasync_run.php executes your job. First, it will check if
you have defined a phase_proceed() method. If you haven't, DataSync will continue by default, but if
you have it will call phase_proceed() and only continue with this job if it is true (it will call
postpone_phase() if phase_proceeed() returns false). Next, it will call the do_phase() method which 
will call the method of the appropriate phase. Next it will call the advance_phase() method which
will move the job on to the next phase in your list. After all the phases are completed successfully,
the job will move into phase "datasync_done", and it will never be called again.

It is important to note that all these methods are overridable in your job type class. If you are
running a really complicated job or want to override default datasync_job methods for any reason,
I recommend you look through datasync.job.inc which is pretty well documented.


install and configuration
--------------------------
See the INSTALL.txt file


todo (for developers only)
--------------
make ds_variable_get('datasync_fail_job_restart', 1) work


Originally contributed by Sony Music
1	; $Id: README.txt,v 1.1.2.4 2009/07/28 20:41:40 andrewlevine Exp $
2
3	description
4	-------------
5	The DataSync module was written to schedule and execute jobs in a much more scalable
6	and reliable way than with cron.php and hook_cron. It allows you to schedule and run
7	module-definable types of jobs on multiple servers in a centralized way.
8	It uses database transactions in order to ensure that no data is corrupted while
9	running multiple jobs at the same time. For this reason, only MySQL is supported and
10	you must ensure all tables that must have consistent data are InnoDB tables (Postgres
11	support is now experimental). This use of transactions allows you to run multiple jobs
12	(on multiple servers, if you want) touching the same database tables at the same time.
13	DataSync keeps track of the status of jobs and what phase they are in, so if a job
14	fails it can intelligently decide how to proceed. Almost all of DataSync's default
15	behavior is overridable when you create your own job types.
16
17	the datasync api
18	-----------------
19	DataSync provides an API to schedule, keep track of, and run your own jobs. To implement
20	the API, you create a "job type" by extending the datasync_job class and registering
21	your own class with the hook_datasync_jobs hook. DataSync will then automatically run
22	your jobs after they are added with a datasync_job_add() call.
23
24	The first thing you have to do to create a job type is create a blank new module. Next,
25	you need to create a file called $JOB_TYPE.job.inc in that module's folder where $JOB_TYPE
26	is the name of your new job type. It should extend the datasync_job class so its definition
27	will be something like this:
28	class my_job_type extends datasync_job {
29
30	Note: You can also extend another already implemented job type instead but the class hierarchy
31	must begin with datasync_job.
32
33	In this class you have to implement, at a minimum, the phase_defs method and a method for each
34	of your phases. The phase_defs method is implemented like this:
35	public function phase_defs() {
36	In this method you just return an array of strings that represent the phases your job has to go
37	through (order is important). DataSync will simply step through these phases, one by one, as
38	fast as it can. So for example, this would be a full phase_defs implementation:
39	public function phase_defs() {
40	return array('begin', 'read_data', 'save_data', 'exit');
41	}
42	Next, you must implement a method for each one of your phases. You can have a minimum of one
43	phase. But in this example you would have to implement:
44	public function begin() {
45	public function read_data() {
46	public function save_data() {
47	public function exit() {
48	Each of these methods should return TRUE on success and FALSE on anything else.
49
50	Next, you need to implement the hook_datasync_job_types hook in the module you have created. So for
51	example, if the module we have created is called my_job_type, our definition would look something
52	like this:
53	function my_job_type_datasync_job_types() {
54	return array(
55	'my_job_type' => array(
56	'human_name' => t('Name of My Job Type'),
57	'description' => t('Description of my job type'),
58	'parent' => 'datasync_job',
59	),
60	);
61	}
62	The key of the array you return must be the same of the class you created, the human_name and description
63	elements are the human readable descriptors of your job type, and parent must be the name of the
64	class you extended (usually datasync_job).
65
66	At this point, assuming you've set up DataSync as specified in INSTALL.txt, when you add a job with:
67	datasync_job_add($YOUR_JOB_TYPE);
68	DataSync should automatically run your job.
69
70	datasync_scheduler
71	-------------------
72	The datasync_scheduler module is included in the datasync module package and provides a user interface
73	to schedule exactly when you want jobs of different types to be created and timed out. It also hooks into
74	the main datasync module (with the datasync_post_jobs hook) and runs/times out these jobs according to
75	the schedule you created. Check the README.txt in that module's folder for more information and about
76	APIs that datasync_scheduler provides.
77
78	datasync_cronalt
79	-------------------
80	Check the README.txt in this module's subdirectory.
81
82
83	If you have any questions, feel free to contact me (Andrew Levine) through my
84	drupal.org user contact page:
85	http://drupal.org/user/49940/contact
86
87
88	extra info on the api:
89	It may also be helpful for you to know how datasync_run.php executes your job. First, it will check if
90	you have defined a phase_proceed() method. If you haven't, DataSync will continue by default, but if
91	you have it will call phase_proceed() and only continue with this job if it is true (it will call
92	postpone_phase() if phase_proceeed() returns false). Next, it will call the do_phase() method which
93	will call the method of the appropriate phase. Next it will call the advance_phase() method which
94	will move the job on to the next phase in your list. After all the phases are completed successfully,
95	the job will move into phase "datasync_done", and it will never be called again.
96
97	It is important to note that all these methods are overridable in your job type class. If you are
98	running a really complicated job or want to override default datasync_job methods for any reason,
99	I recommend you look through datasync.job.inc which is pretty well documented.
100
101
102	install and configuration
103	--------------------------
104	See the INSTALL.txt file
105
106
107	todo (for developers only)
108	--------------
109	make ds_variable_get('datasync_fail_job_restart', 1) work
110
111
112	Originally contributed by Sony Music