1 Introduction
2 ------------
3
4 ConfluenceConverter is a distribution of software that converts exported data
5 from Confluence wiki instances, provided in the form of an XML file, to a
6 collection of wiki pages and resources that can be imported into a MoinMoin
7 instance as a page package.
8
9 Migration Activities
10 --------------------
11
12 The following activities are involved in a migration from Confluence to
13 MoinMoin:
14
15 * Export of Confluence content
16 * Conversion of Confluence content to MoinMoin content
17 * Confluence page identifier extraction and mapping to MoinMoin identifiers
18 * Acquisition of Confluence user profile details
19 * Installation of MoinMoin
20 * Initialisation of a MoinMoin wiki instance
21 * Import of MoinMoin content into the new wiki instance
22 * Installation of MoinMoin extensions
23 * Initialisation of user profiles in MoinMoin
24 * Installation of scripts and identifier mappings
25 * Filesystem permission adjustments
26
27 Prerequisites
28 -------------
29
30 ConfluenceConverter requires a library called xmlread that can be found at the
31 following location:
32
33 http://hgweb.boddie.org.uk/xmlread
34
35 The xmlread.py file from the xmlread distribution can be copied into the
36 ConfluenceConverter directory.
37
38 ConfluenceConverter also requires access to the MoinMoin.wikiutil module found
39 in the MoinMoin distribution.
40
41 The moinsetup program is highly recommended for the installation of page
42 packages and the management of MoinMoin wiki instances:
43
44 http://moinmo.in/ScriptMarket/moinsetup
45
46 If moinsetup is not being used, the page package installer documentation
47 should be consulted:
48
49 http://moinmo.in/HelpOnPackageInstaller
50
51 To read Confluence user profiles on live Confluence sites using the
52 get_profiles.py program, the libxml2dom library is required:
53
54 http://hgweb.boddie.org.uk/libxml2dom
55
56 MoinMoin Prerequisites
57 ----------------------
58
59 The page package installer does not preserve user information or the last
60 modified time when installing page revisions. This can be modified by applying
61 a patch to MoinMoin as follows while at the top level of the MoinMoin source
62 distribution:
63
64 patch -p1 $CCDIR/patches/patch-moin-1.9-MoinMoin-packages.diff
65
66 Here, CCDIR is the path to the top level of this source distribution where
67 this README.txt file is found.
68
69 Wiki Content Prerequisites
70 --------------------------
71
72 For the output of the converter, the following MoinMoin extensions are
73 required:
74
75 http://moinmo.in/ParserMarket/ImprovedTableParser
76 http://hgweb.boddie.org.uk/MoinSupport
77 http://moinmo.in/MacroMarket/Color2
78
79 In addition, extensions are provided in this distribution to support various
80 Confluence features, notably comments on pages. These extensions are installed
81 as follows:
82
83 python moinsetup.py -m install_actions $CCDIR/actions
84 python moinsetup.py -m install_macros $CCDIR/macros
85 python moinsetup.py -m install_theme_resources $CCDIR
86 python moinsetup.py -m edit_theme_stylesheet screen.css includecomments.css
87 python moinsetup.py -m edit_theme_stylesheet print.css includecomments.css
88
89 Additional Software
90 -------------------
91
92 PDF export support requires the ExportPDF action:
93
94 http://moinmo.in/ActionMarket/ExportPDF
95
96 This in turn requires Apache FOP for PDF production using XSL-FO:
97
98 http://xmlgraphics.apache.org/fop/
99
100 (On Debian systems, the fop package provides this tool.)
101
102 To produce XSL-FO from DocBook output, xsltproc is required from the libxslt
103 distribution:
104
105 http://xmlsoft.org/XSLT/
106
107 (On Debian systems, the xsltproc package provides this tool.)
108
109 And DocBook output requires the DocBook resources to be installed, described
110 in the following guide:
111
112 http://www.sagehill.net/docbookxsl/ToolsSetup.html
113
114 (On Debian systems, the docbook-xsl package provides these resources.)
115
116 Quick Start
117 -----------
118
119 Given an XML export archive file for a Confluence wiki instance (in the
120 example below, the file is called COM-123456-789012.zip), the following
121 command can be used to prepare a page package for MoinMoin:
122
123 python convert.py COM-123456-789012.zip COM
124
125 In addition to the filename, a workspace name is required. Confluence appears
126 to require a workspace as a container for collections of pages, but this also
127 permits us to selectively import parts of a wiki into MoinMoin. If attachments
128 were included in the export from Confluence, these will be imported into the
129 page package.
130
131 The result of the above command will be a directory having the same name as
132 the chosen workspace, together with a zip archive for that directory's
133 contents. Thus, the above command would produce a directory called COM and an
134 archive called COM.zip.
135
136 To import the result, use moinsetup as follows:
137
138 python moinsetup.py -m install_page_package COM.zip
139
140 This requires a suitable moinsetup.cfg file in the working directory.
141
142 Importing Many Workspaces
143 -------------------------
144
145 Where more than one namespace is to be imported, the page packages should be
146 merged so that the resulting history information is ordered correctly.
147
148 To merge packages, use a command of the following form:
149
150 python merge.py OUT COM.zip DEV.zip DOC.zip SEC.zip
151
152 A directory called OUT and a page package called OUT.zip will be produced. The
153 latter can then be imported into MoinMoin as described above.
154
155 Mappings from Identifiers to Pages
156 ----------------------------------
157
158 Confluence uses numbers to label content revisions, and links to Confluence
159 sites sometimes use these numbers instead of a readable page name. MoinMoin,
160 meanwhile, only uses page names and has no external numeric identifier scheme.
161 Consequently, it is necessary to produce a mapping from Confluence identifiers
162 to MoinMoin page names. In addition to numeric identifiers, Confluence also
163 provides "tiny URLs" which are an alphanumeric encoding of the numeric
164 identifiers.
165
166 To generate mappings for the Confluence content, use the mappings script as
167 follows:
168
169 tools/mappings.sh COM
170
171 Here, COM is a directory name containing converted Confluence content,
172 corresponding to a space name in the original Confluence wiki. More than one
173 space name can be used to generate a complete mapping for a site.
174
175 The following files are generated:
176
177 * mapping-id-to-page.txt
178 * mapping-tiny-to-id.txt
179 * mapping-tiny-to-page.txt
180
181 The most useful of these is the first as it includes all the necessary
182 information provided by the arbitrary mapping from identifiers to page names.
183 The second mapping merely converts the "tiny URLs" to identifiers, which can
184 be done by applying an algorithm without any external knowledge of the wiki
185 structure. The third mapping is provided as a convenience, combining the "tiny
186 URL" conversion and the arbitrary mapping to page names.
187
188 Translating Requests Using the Mappings
189 ---------------------------------------
190
191 Where Web server facilities such as RewriteMap are available for use, the
192 first and third mapping files can be used directly. See the Apache
193 documentation for details of RewriteMap:
194
195 http://httpd.apache.org/docs/2.4/rewrite/rewritemap.html
196
197 Otherwise, it is more likely that the first file is used by a program that can
198 perform a redirect to the appropriate wiki page, and the "tiny URL" decoding
199 is also done by this program when deployed in a suitable location to receive
200 such requests. To support this, the following resources are provided:
201
202 * scripts/redirect.py
203 * config/mailmanwiki-redirect
204
205 The latter configuration file should be combined with the Web server
206 configuration file such that the appropriate aliases are able to capture
207 requests and invoke the redirect.py script before the main wiki aliases are
208 consulted. The script itself should be placed in a suitable filesystem
209 location, and the mapping-id-to-page.txt file should be placed alongside it,
210 or it should be placed in a different location and the MAPPING_ID_TO_PAGE
211 variable changed in the script to refer to this different location.
212
213 Supporting Confluence Action URLs
214 ---------------------------------
215
216 Besides the "viewpage" action mapping identifiers to pages (covered by the
217 mapping described above), some other action URLs may be used in wiki content
218 and must either be translated or supported using redirects. Since external
219 sites may also employ such actions, a redirect strategy perhaps makes more
220 sense. To support this, the following resources are involved:
221
222 * scripts/dashboard.py
223 * scripts/redirect.py
224 * scripts/search.py
225 * config/mailmanwiki-redirect
226
227 The latter configuration file is also involved in identifier-to-page mapping,
228 but in this case it causes requests to the "dashboard", "doexportpage" and
229 "dosearchsite" actions to be directed to the dashboard.py, redirect.py and
230 search.py scripts respectively.
231
232 The dashboard.py script merely redirects requests to the root of the site,
233 thus assuming that the front page is configured to show dashboard-like
234 information.
235
236 The redirect.py script, apart from supporting identifier-to-page redirects,
237 also supports PDF page exports since the "doexportpage" action uses
238 identifiers to indicate which page is to be exported.
239
240 The search.py script redirects search requests in a suitable form to the
241 MoinMoin "fullsearch" action.
242
243 Identifying and Migrating Users
244 -------------------------------
245
246 Confluence export archives do not contain user profile information, but page
247 versions are marked with user identifiers. Therefore, a list of user
248 identifiers can be obtained by running a script extracting these identifiers.
249 The following command writes to standard output the users involved with
250 editing the wiki in four different spaces (exported to four directories):
251
252 tools/users.sh COM DEV DOC SEC
253
254 This output can be edited and then passed to a program which fetches other
255 profile details as follows:
256
257 tools/users.sh COM DEV DOC SEC > users.txt # for editing
258 cat users.txt | tools/get_profiles.py http://wiki.list.org/
259
260 If no users are to be removed in migration, the following command could be
261 issued:
262
263 tools/users.sh COM DEV DOC SEC | tools/get_profiles.py http://wiki.list.org/
264
265 The get_profiles.py program needs to be told the URL of the original
266 Confluence site. Note that it accesses the site at a default rate of around
267 one request per second; a different delay between requests can be specified
268 using an additional argument.
269
270 The output of the get_profiles.py program can be passed to another program
271 which adds users to MoinMoin, and so the following commands can be used:
272
273 cat users.txt \
274 | tools/get_profiles.py http://wiki.list.org/ \
275 | tools/addusers.py wiki
276
277 And using one single command:
278
279 tools/users.sh COM DEV DOC SEC \
280 | tools/get_profiles.py http://wiki.list.org/ \
281 | tools/addusers.py wiki
282
283 The addusers.py program needs to be told the directory containing the wiki
284 configuration.
285
286 Output Structure
287 ----------------
288
289 The structure of a converted workspace is a directory hierarchy containing the
290 following directories:
291
292 * pages (a collection of directories defining each page or content item,
293 corresponding to Page, Comment and BlogPost elements in the XML
294 exported from Confluence)
295
296 * versions (a collection of files, each defining a revision or version of
297 some content, corresponding to BodyContent elements in the XML
298 exported from Confluence)
299
300 Each page directory contains the following things:
301
302 * pagetype (either "Page", "Comment" or "BlogPost")
303
304 * manifest (a list of version entries in a format similar to the MoinMoin
305 page package manifest format)
306
307 * attachments (a list of attachment version entries in a format similar to
308 the MoinMoin page package manifest format)
309
310 * pagetitle (an optional page title imposed on the page by another content
311 item)
312
313 * children (a list of child page names defined for the page)
314
315 * comments (a list of creation date plus comment page identifier pairs)
316
317 In the output structure, content items such as comments are represented as
318 pages and each reference a content version. Since comments will ultimately be
319 represented as subpages of some parent page, they will have a pagetitle file
320 in their directory with an appropriate subpage name written according to the
321 parent page's name and comment details.
322
323 Troubleshooting
324 ---------------
325
326 The page package import activity in particular can be a source of problems.
327 Generally, any error occurring when attempting to import a package is likely
328 to be due to insufficient privileges when writing to the pages directory of a
329 wiki or to its edit-log file.
330
331 The moinsetup software can generate scripts that set the ownership of wiki
332 files or apply ACLs (access control lists) to those files in order to make
333 access to wiki data more convenient. Where the ownership of the files must be
334 set (to www-data or nobody), the import step can be run as that user given
335 sufficient privileges. However, the easiest solution is to apply ACLs, thus
336 allowing the user who created the wiki to retain write access to it.
337
338 Contact, Copyright and Licence Information
339 ------------------------------------------
340
341 The current Web page for ConfluenceConverter at the time of release is:
342
343 http://hgweb.boddie.org.uk/ConfluenceConverter
344
345 Copyright and licence information can be found in the docs directory - see
346 docs/COPYING.txt and docs/LICENCE.txt for more information.