Forum: CAT Tools Technical Help
Topic: Customization of Segmentation (all tools)
Poster: Stanislav Okhvat
Post title: Re: Customization of Segmentation (all tools)
Hello Helen,
With tab characters, it is pretty easy to configure your CAT tool so that there is no segmentation on tab characters. Some CAT tools such as memoQ do not perform segmentation on tab characters by default, others like Memsource do.
With paragraph breaks (hard returns) it is more difficult. A paragraph is a distinct structural unit so when a CAT tool segments a document, it first breaks the text into structural units (e.g., in Word these are paragraphs, in HTML these are div, p and similar structural tags, etc.). After the text is broken down into structural units, each piece is segmented according to segmentation rules. For this reason it is impossible to avoid segmentation on paragraph breaks -- you can create a segmentation rule for this, but it will never work. Note, however, that it is possible to avoid segmentation on line breaks (soft returns). In Word you can use Unbreaker for Word (part of a Word add-in) to remove hard returns in a semi-automatic way before the document is imported into the CAT tool, thus avoiding the above issue.
Regarding the ellipsis (…), it is easy enough to prevent segmentation on this character.
Memsource uses segmentation rules in SRX format. For Word, it segments on tab characters by default (it is easy to set up a default rule for a specific source language which will avoid segmenting on tab characters). By default, Memsource does not segment on ellipsis (Unicode character U+2026). You can customize Memsource and other CAT tools so that they segment only on specific punctuation marks, e.g. on . as in your case.
Best regards,
Stanislav Okhvat
TransTools – Useful tools for every translator
Topic: Customization of Segmentation (all tools)
Poster: Stanislav Okhvat
Post title: Re: Customization of Segmentation (all tools)
Hello Helen,
With tab characters, it is pretty easy to configure your CAT tool so that there is no segmentation on tab characters. Some CAT tools such as memoQ do not perform segmentation on tab characters by default, others like Memsource do.
With paragraph breaks (hard returns) it is more difficult. A paragraph is a distinct structural unit so when a CAT tool segments a document, it first breaks the text into structural units (e.g., in Word these are paragraphs, in HTML these are div, p and similar structural tags, etc.). After the text is broken down into structural units, each piece is segmented according to segmentation rules. For this reason it is impossible to avoid segmentation on paragraph breaks -- you can create a segmentation rule for this, but it will never work. Note, however, that it is possible to avoid segmentation on line breaks (soft returns). In Word you can use Unbreaker for Word (part of a Word add-in) to remove hard returns in a semi-automatic way before the document is imported into the CAT tool, thus avoiding the above issue.
Regarding the ellipsis (…), it is easy enough to prevent segmentation on this character.
Memsource uses segmentation rules in SRX format. For Word, it segments on tab characters by default (it is easy to set up a default rule for a specific source language which will avoid segmenting on tab characters). By default, Memsource does not segment on ellipsis (Unicode character U+2026). You can customize Memsource and other CAT tools so that they segment only on specific punctuation marks, e.g. on . as in your case.
Best regards,
Stanislav Okhvat
TransTools – Useful tools for every translator