1 Introduction
2 ------------
3
4 ConfluenceConverter is a distribution of software that converts exported data
5 from Confluence wiki instances, provided in the form of an XML file, to a
6 collection of wiki pages and resources that can be imported into a MoinMoin
7 instance as a page package.
8
9 Prerequisites
10 -------------
11
12 ConfluenceConverter requires a library called xmlread that can be found at the
13 following location:
14
15 http://hgweb.boddie.org.uk/xmlread
16
17 The xmlread.py file from the xmlread distribution can be copied into the
18 ConfluenceConverter directory.
19
20 ConfluenceConverter also requires access to the MoinMoin.wikiutil module found
21 in the MoinMoin distribution.
22
23 The moinsetup program is highly recommended for the installation of page
24 packages and the management of MoinMoin wiki instances:
25
26 http://moinmo.in/ScriptMarket/moinsetup
27
28 If moinsetup is not being used, the page package installer documentation
29 should be consulted:
30
31 http://moinmo.in/HelpOnPackageInstaller
32
33 To read Confluence user profiles on live Confluence sites using the
34 get_profiles.py program, the libxml2dom library is required:
35
36 http://hgweb.boddie.org.uk/libxml2dom
37
38 MoinMoin Prerequisites
39 ----------------------
40
41 The page package installer does not preserve user information when installing
42 page revisions. This can be modified by applying a patch to MoinMoin as
43 follows while at the top level of the MoinMoin source distribution:
44
45 patch -p1 $CCDIR/patches/patch-moin-1.9-MoinMoin-packages.diff
46
47 Here, CCDIR is the path to the top level of this source distribution where
48 this README.txt file is found.
49
50 Wiki Content Prerequisites
51 --------------------------
52
53 For the output of the converter, the following MoinMoin extensions are
54 required:
55
56 http://moinmo.in/ParserMarket/ImprovedTableParser
57 http://hgweb.boddie.org.uk/MoinSupport
58 http://moinmo.in/MacroMarket/Color2
59
60 In addition, extensions are provided in this distribution to support various
61 Confluence features, notably comments on pages. These extensions are installed
62 as follows:
63
64 python moinsetup.py -m install_actions $CCDIR/actions
65 python moinsetup.py -m install_macros $CCDIR/macros
66 python moinsetup.py -m install_theme_resources $CCDIR
67 python moinsetup.py -m edit_theme_stylesheet screen.css includecomments.css
68 python moinsetup.py -m edit_theme_stylesheet print.css includecomments.css
69
70 Quick Start
71 -----------
72
73 Given an XML export archive file for a Confluence wiki instance (in the
74 example below, the file is called COM-123456-789012.zip), the following
75 command can be used to prepare a page package for MoinMoin:
76
77 python convert.py COM-123456-789012.zip COM
78
79 In addition to the filename, a workspace name is required. Confluence appears
80 to require a workspace as a container for collections of pages, but this also
81 permits us to selectively import parts of a wiki into MoinMoin. If attachments
82 were included in the export from Confluence, these will be imported into the
83 page package.
84
85 The result of the above command will be a directory having the same name as
86 the chosen workspace, together with a zip archive for that directory's
87 contents. Thus, the above command would produce a directory called COM and an
88 archive called COM.zip.
89
90 To import the result, use moinsetup as follows:
91
92 python moinsetup.py -m install_page_package COM.zip
93
94 This requires a suitable moinsetup.cfg file in the working directory.
95
96 Mappings from Identifiers to Pages
97 ----------------------------------
98
99 Confluence uses numbers to label content revisions, and links to Confluence
100 sites sometimes use these numbers instead of a readable page name. MoinMoin,
101 meanwhile, only uses page names and has no external numeric identifier scheme.
102 Consequently, it is necessary to produce a mapping from Confluence identifiers
103 to MoinMoin page names. In addition to numeric identifiers, Confluence also
104 provides "tiny URLs" which are an alphanumeric encoding of the numeric
105 identifiers.
106
107 To generate mappings for the Confluence content, use the mappings script as
108 follows:
109
110 tools/mappings.sh COM
111
112 Here, COM is a directory name containing converted Confluence content,
113 corresponding to a space name in the original Confluence wiki. More than one
114 space name can be used to generate a complete mapping for a site.
115
116 The following files are generated:
117
118 * mapping-id-to-page.txt
119 * mapping-tiny-to-id.txt
120 * mapping-tiny-to-page.txt
121
122 The most useful of these is the first as it includes all the necessary
123 information provided by the arbitrary mapping from identifiers to page names.
124 The second mapping merely converts the "tiny URLs" to identifiers, which can
125 be done by applying an algorithm without any external knowledge of the wiki
126 structure. The third mapping is provided as a convenience, combining the "tiny
127 URL" conversion and the arbitrary mapping to page names.
128
129 Translating Requests Using the Mappings
130 ---------------------------------------
131
132 Where Web server facilities such as RewriteMap are available for use, the
133 first and third mapping files can be used directly. See the Apache
134 documentation for details of RewriteMap:
135
136 http://httpd.apache.org/docs/2.4/rewrite/rewritemap.html
137
138 Otherwise, it is more likely that the first file is used by a program that can
139 perform a redirect to the appropriate wiki page, and the "tiny URL" decoding
140 is also done by this program when deployed in a suitable location to receive
141 such requests. To support this, the following resources are provided:
142
143 * scripts/redirect.py
144 * config/mailmanwiki-redirect
145
146 The latter configuration file should be combined with the Web server
147 configuration file such that the appropriate aliases are able to capture
148 requests and invoke the redirect.py script before the main wiki aliases are
149 consulted. The script itself should be placed in a suitable filesystem
150 location, and the mapping-id-to-page.txt file should be placed alongside it,
151 or it should be placed in a different location and the MAPPING_ID_TO_PAGE
152 variable changed in the script to refer to this different location.
153
154 Supporting Confluence Action URLs
155 ---------------------------------
156
157 Besides the "viewpage" action mapping identifiers to pages (covered by the
158 mapping described above), some other action URLs may be used in wiki content
159 and must either be translated or supported using redirects. Since external
160 sites may also employ such actions, a redirect strategy perhaps makes more
161 sense. To support this, the following resources are involved:
162
163 * scripts/search.py
164 * config/mailmanwiki-redirect
165
166 The latter configuration file is also involved in identifier-to-page mapping,
167 but in this case it causes requests to the "dosearchsite" action to be
168 directed to the search.py script, which in turn redirects the request in a
169 suitable form to the MoinMoin "fullsearch" action.
170
171 Identifying and Migrating Users
172 -------------------------------
173
174 Confluence export archives do not contain user profile information, but page
175 versions are marked with user identifiers. Therefore, a list of user
176 identifiers can be obtained by running a script extracting these identifiers.
177 The following command writes to standard output the users involved with
178 editing the wiki in four different spaces (exported to four directories):
179
180 tools/users.sh COM DEV DOC SEC
181
182 This output can be edited and then passed to a program which fetches other
183 profile details as follows:
184
185 tools/users.sh COM DEV DOC SEC > users.txt # for editing
186 cat users.txt | tools/get_profiles.py http://wiki.list.org/
187
188 If no users are to be removed in migration, the following command could be
189 issued:
190
191 tools/users.sh COM DEV DOC SEC | tools/get_profiles.py http://wiki.list.org/
192
193 The get_profiles.py program needs to be told the URL of the original
194 Confluence site. Note that it accesses the site at a default rate of around
195 one request per second; a different delay between requests can be specified
196 using an additional argument.
197
198 The output of the get_profiles.py program can be passed to another program
199 which adds users to MoinMoin, and so the following commands can be used:
200
201 cat users.txt \
202 | tools/get_profiles.py http://wiki.list.org/ \
203 | tools/addusers.py wiki
204
205 And using one single command:
206
207 tools/users.sh COM DEV DOC SEC \
208 | tools/get_profiles.py http://wiki.list.org/ \
209 | tools/addusers.py wiki
210
211 The addusers.py program needs to be told the directory containing the wiki
212 configuration.
213
214 Output Structure
215 ----------------
216
217 The structure of a converted workspace is a directory hierarchy containing the
218 following directories:
219
220 * pages (a collection of directories defining each page or content item,
221 corresponding to Page, Comment and BlogPost elements in the XML
222 exported from Confluence)
223
224 * versions (a collection of files, each defining a revision or version of
225 some content, corresponding to BodyContent elements in the XML
226 exported from Confluence)
227
228 Each page directory contains the following things:
229
230 * pagetype (either "Page", "Comment" or "BlogPost")
231
232 * manifest (a list of version entries in a format similar to the MoinMoin
233 page package manifest format)
234
235 * attachments (a list of attachment version entries in a format similar to
236 the MoinMoin page package manifest format)
237
238 * pagetitle (an optional page title imposed on the page by another content
239 item)
240
241 * children (a list of child page names defined for the page)
242
243 * comments (a list of creation date plus comment page identifier pairs)
244
245 In the output structure, content items such as comments are represented as
246 pages and each reference a content version. Since comments will ultimately be
247 represented as subpages of some parent page, they will have a pagetitle file
248 in their directory with an appropriate subpage name written according to the
249 parent page's name and comment details.
250
251 Troubleshooting
252 ---------------
253
254 The page package import activity in particular can be a source of problems.
255 Generally, any error occurring when attempting to import a package is likely
256 to be due to insufficient privileges when writing to the pages directory of a
257 wiki or to its edit-log file.
258
259 The moinsetup software can generate scripts that set the ownership of wiki
260 files or apply ACLs (access control lists) to those files in order to make
261 access to wiki data more convenient. Where the ownership of the files must be
262 set (to www-data or nobody), the import step can be run as that user given
263 sufficient privileges. However, the easiest solution is to apply ACLs, thus
264 allowing the user who created the wiki to retain write access to it.
265
266 Contact, Copyright and Licence Information
267 ------------------------------------------
268
269 The current Web page for ConfluenceConverter at the time of release is:
270
271 http://hgweb.boddie.org.uk/ConfluenceConverter
272
273 Copyright and licence information can be found in the docs directory - see
274 docs/COPYING.txt and docs/LICENCE.txt for more information.