1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
|
Tinbuild2 - automatic building of LibreOffice
=============================================
tinbuild2 is an evolution of tinbuild - it periodically pulls from the
LibreOffice git, builds the thing, and optionally mails those who broke the
build, or uploads the build results to http://dev-builds.libreoffice.org/daily/
Short version
-------------
<profile_name> is a name you chose for you profile... usually use 'master' for a tinderbox
that run the master branch. but you can call it anything you want.
You can create different profile that do different kind of build, following the same procedure.
Read http://wiki.documentfoundation.org/Development/Tinderbox .
# clone tinbuild
git clone git://gerrit.libreoffice.org/buildbot /lo/buildbot
# clone LibreOffice
git clone git://gerrit.libreoffice.org/core /lo/core
# make the LibreOffice build
cd /lo/core
./autogen.sh [ which ever argument are needed ]
make
make dev-install
# after the above succeeded, configure buildbot
mkdir -p ~/.tinbuild/config
mkdir -p ~/.tinbuild/metadata/<profile_name>
cat > ~/.tinbuild/config/<profile_name>.cfg <<EOF
SMTPHOST=<your value>
SMTPUSER=<your value>
SMTPPW=<your value>
TINDER_NAME=<your value> # [ see https://wiki.documentfoundation.org/Development/Tinderbox for naming conventions ]
TINDER_ID=<id> # [ see https://wiki.documentfoundation.org/Development/Tinderbox for how to get an ID ]
OWNER=<email address of the owner of this TB>
METADATA_DIR=~/.tinbuild/meta/<profile_name> # this can be any existing directory you want
# if METADATA_DIR is not specified the tinderbox metadata are stored in the core repo itself
GERRIT_PLATFORM=(MAC|LINUX|WINDOWS) # platform this TB is building when doing gerrit path verification
EOF
# save you autogen configuration.
mkdir -p ~/.tinbuild/autogen
cp /lo/core/autogen.lastrun ~/.tinbuild/autogen/<prodile_name>.autogen
# check the configuration
cd /lo/core
/lo/buildbot/bin/tinbuild2 -p <profile_name> -c
# prime the tinderbox
cd /lo/core
/lo/buildbot/bin/tinbuild2 -p <profile_name> -z
# and if that succeeds, start the periodical building
cd /lo/core
/lo/buildbot/bin/tinbuild2 -m all -p <profile_name>
# to also build gerrit patches
/lo/buildbot/bin/tinbuild2 -m all -p <prodile_name> --mode=fair
Configuration
-------------
The configuration of tinbuild2 is stored in ~/.tinbuild/*
tinbuild2 is 'profile' based.
each profile comport a config file in
~/.tinbuild/config/<profile_name>.cfg
and an assocaited autogen configuration for the build
located at
~/.tinbuild/autogen/<profile_name>.autogen
optionnaly each profile can source an 'extension/override' script
located at
~/.tinbuild/phases/<profile_name>.sh
that file, if it exsist will be sourced in tinbuild2 script.
tinbuild2 maintain some metadata information in order to keep track ot the state
of affair for a given profile.
by default these files are sotre in the directory of the core repo, but
if you specify a METADATA_DIR in the configuration of the profile then
that directory will be used to store these files.
Note: the METADATA_DIR, if specified, must exist.
Using a METADATA_DIR allow to keep the repo clean, allow to use r/o repo,
all to use git clean on the repo without risk of destroying the state of the tinderbox.
config file
-----------
the config file for a profile is located at
~/.tindbuild/config/<profile_name>.cfg
The following parameter can od must be specified.
Mandatory parameters:
SMTPHOST : hostname of the smtp service to send email.
SMTPUSER : user to pass to the smtp server to send email.
SMTPPW : password for the smp service
SMTPUSER and SMTPPW may be ommited if your smtp service connection without login. That is usually not the case.
TINDER_NAME : name of yur tinderbox, please refer to
https://wiki.documentfoundation.org/Development/Tinderbox
for tinderbox naming conventions
TINDER_ID : numerical id of your tinderbox. Please refer to
https://wiki.documentfoundation.org/Development/Tinderbox
for how to assign an id to your tinderbox
OWNER : email address of the owner of this tinderbox.
Optional parameters
METADATA_DIR : directory where tinbuild2 will store the metadata information about the state of the tinderbox
if specified, the directory must exist (it is _not_ created by tinbuild2)
if this variable is not specified, then the metadata fiels are store in the root of
the core repository.
ARTIFACTDIR : git repo where to deliver binary build.
tinbuild2 can collect binary build to form a bibisect git repo
this parameter is mandatody if you specifyc the '-x' command line argument
when running tinbuild2
BIBISECT_GC : Y/N. In case of collection of binary artifact, make tinbuild2 to run a git gc command after
the collection of each new binary version
BIBSECT_PUSH : Y/N. In case of collection of binary artifact, make tinbuild2 push the bibisect repo to
its remote 'origin' after the collection of each new binary version
GERRIT_PLATFORM : MAC/LINUX/WINDOWS. when using tinbuild2 to do gerrit patch verification,
indicate which class of build that tinderbox build.
This parameter may be needed if the auto-termination that tinbuild2 do
using 'uname' is uncorrect, or possibly in the future to support cross-compile
tinderbox.
PAUSE_SECONDS : <integer>. Duration, in seconds, to pause after each build, before trying to do another one.
The default value is 900 seconds (15 minutes).
For a dedicated tindebox machine, this should probably be 60 or less.
command-line syntax:
tinbuild2 -p <profile_name> [options]
-0 Do incremental build; that is do not invoke the clean phase
between build. Not recommended for any mode of operation that involve
gerrit. In general that should be reserved to pure tinderbox mode
and that tinderbox need to be monitored closely to detect repeated failure
due to problem with incremental build.
-a <email> for a run that do not report to the tinderbox server, but
is used to to a 'private' test of a particular branch -- like
a feature branch for instance -- this specify the email of
the 'author' to notify of the tinderbox activity.
this allow you to run a tinderbox to test a feature branch
and automatically send the result to a given person.
This work in conjonction with '-m author'. See below.
-h print help.
-c Execute a 'sanity check'. Verify that the configuration is sane
and send a test email to the owner, to verify that the smtp mechanism
works.
-g <gerrit_ref> To a test build using the given gerrit ref to checkout a specific commit
A gerrit ref is in the form of : refs/changes/<n>/<m>/<p>
Message are sent in the Review section of the assocaited gerrit patch
to indicate that a build as started and to indicate the result of the build.
This option imply a one shot build. The tinbuild2 terminate after that one build.
-i run with ionice -c3
-k To not overrite autogen.lastrun if present. This allow to use -z
to do a quick test of the current config
-m all|tb|owner|author|none Set the email level in regular tindeebox run
This does not apply to gerrit run or priming run.
all : send email to the tinderbox server, to the owner and to all the committer
involved in a failed build.
tb : send email to the tinderbox server and the owner
owner : send only tinderbox to the owner.
author : send email to the specified author (see -a above)
none : do not send email at all.
--mode tb|gerrit|tb-gerrit|gerrit-tb|fair Mode of operation
tb : normal tinderbox mode. Wait on new commit to show on core
and build as new commits are found...
gerrit : gerrit patch verification mode. Query gerrit for patch to test-build
build them and report back, and repeat.
tb-gerrit: a mix of tb and gerrit, where if there is no new commit to be
build in tb mode, we attempt to get a task from gerrit
This mode favor the tb mode, in the sens that gerrit task
are only attempted if there is nothing left to do in tb mode
gerrit-tb : like tb-gerrit, except the the scheduling favor gerrit
in priority.
fair : dual mode tb and gerrit like tb-gerrit, except that the scheduling
priority is given alternatively to tb and gerrit.
The default mode is tb. --mode is not to be used conjunction with -z or -g
which are meant to trigger one-shot build.
-n Run with 'nice'
-r <bandwidth> Indicate that we want to push daily upload with a maximum upload
bandwidth as indicated in KB/s
This will attempt to upload a build at most once a day.
This require that you have obtained ssh keys to be able
to upload to dev-build.libreoffice.org
-t Run the full suite of test after a sucessfull build
-v Print more message while tinbuild2 is running
-x Push sucessfull binary build, at most once a day to a bibisect repo
This require that ARTIFACTDIR be specified in the configuration of the
profile and point to a bibisect git repo.
-z Run a 'Priming' build. In order for tinbuild2 to be able to notify
the appropriate subset of committers in case of failure, we need
a sucessfull 'reference' build.
You are required to run a sucessfull primer build before you can
run tinbuild2 in a loop.
This is required even if you do not actually email committers with failures.
phases customization
--------------------
The design is loosely inspired from Gentoo's portage. the idea is to use
the ability of bash to redefine function.
so the function to build the product, do_build, is implemented with
tb_call()
{
declare -F "$1" > /dev/null && $1
}
phase()
{
local f=${1}
for x in {pre_,do_,post_}${f} ; do
tb_call ${x}
done
}
do_build()
{
retval=0
for p in autogen clean make test push ; do
echo "call $p"
phase $p
done
}
so essentially the following functions will be called if implemented
pre_autogen, do_autogen, post_autogen, pre_clean, do_clean, post_clean,
pre_make, do_make, post_make, pre_test, do_test, post_test, pre_push, do_push,
post_push
Note that the implementation of these function need to test ${retval} to find
out if a step before them had failed. if so they may decide to do nothing
(most cases) or may decide to still do something.
reciprocally, every function should put a non-zero value in retval in case of
failure.
Now, the base function are implemented in tinbuild_phases.sh
but these function can be overrided by a user-implementation,
typically store in ~/.tinbphuid/phases/<profile_name>.sh
which is sourced by tinbuild2 if it exists.
In the same spirit, platform dependent hack can be done in tinbuild_internals_<uname>.sh
The core default is meant to run on Linux, using bash and all the GNU tools that
are expected to normally be there on a Linux box.
Other platform will re-implementent incompatible stuff in their own .sh
|
