Edit template: Scripting to transform documents before importing and after exporting
Experts only: Don't proceed if you're not familiar with scripts, programs, services, and users in Windows. If you don't have this experience, you need to call an expert, possibly an IT specialist.
Some documents must be transformed before you can import them into memoQ. This is done in project templates, through scripting.
Can't do this without templates: You can do this in projects created from project templates only.
When it's already in a project, it works like this:
- You import a document in the project.
- memoQ automatically runs the document through a program. That produces another, transformed, document. This step is called pre-processing.
- The transformed document is imported in the project.
- You work on the transformed document.
- When you export the document, you'll get a transformed document with the translations.
- Then memoQ runs the exported document through another program, which transforms it back to the original format. This step is called post-processing.
Here's an example:
This Help documentation is written in a program that produces the Help pages in XML format. But if you import these XML files directly, the segments - sentences - will be fragmented, and you need to join every second segment with the next one to get sensible sentences.
There's a program that turns these XML files into slightly different XML files, where the fragments are already joined together. We import and translate these slightly different XML files. After they're exported, they must be transformed back, so that the authoring program can open them again. So they are run through another program that splits the segments apart again into the fragments. (Technically speaking, the same program is run with different arguments, but it could just as well be a completely different one.)
You need to set this up in a project template. Then the transformation will work in all projects that you create from this project template.
Set up your server first: If you need this for online projects, you must set up your server first. This is described in the What can you do? section, later in this topic.
You need to write or acquire the program: memoQ doesn't come with programs that transform documents. Either you need to write these, or acquire them from a programmer. Or, you can turn to memoQ Professional Services, who will write it for you for a fee.
To prepare or touch up text documents, use the built-in Find and replace tool: To learn more, see the topic about the Find and replace script.
How to get here
Open Resource console. Choose Project templates.
To set up a template for online projects: Use an online template for this. Choose your memoQ server at the top of Resource console. Click Select.
Select the project template that you want to set up with import and export scripts. Right-click the selection, and click Edit.
In the Edit project template window, choose Automated actions. Click the Scripts before import tab to set up the transformation that runs before the documents are imported. Click the Scripts after export tab to set up the transformation that runs after the documents are exported. You will probably need both.
What can you do?
The scripts that transform your documents are programs. On a server, not everyone is allowed to run programs. Normally, memoQ server isn't allowed to run programs - for security reasons.
Before you run scripts in an online project, you need to allow memoQ server to run them.
Must be an administrator and must be an expert: You can't follow the steps below if you're not an administrator of the memoQ server computer, and you're not completely aware of the things you're about to do.
You don't do this from memoQ - set aside the memoQ window until you finish these steps.
- Through Remote Desktop, sign in to the memoQ server computer as an administrator. (That is, you must be an administrator of the entire machine, not just the memoQ server program.)
- Open Control Panel. Choose Programs and Features. Select memoQ server. Click Change. The memoQ server deployment tool opens.
- In the memoQ server section of the Deployment program, click Advanced, and then choose Configure custom code execution. The Custom code execution window appears.
- As a rule, choose Virtual service account here. Click Next. Never use a built-in account. You may use a specific account under the This account radio button. Use a specific account only if the account must be a domain account, or your script must access another computer on the network.
- The deployment tool checks if the settings are valid. You must be an administrator of the server computer, and the selected account must be able to log on as a service. If you see two green tick marks in the window, click Next. If there is at least one error, click Back, and change the settings.
- The deployment tool sets up memoQ server for running the scripts and programs. This may take a while. When you see the Settings have been saved successfully message, click Finish.
- A warning appears that you must restart memoQ server. Click OK.
- In the deployment tool, click Stop memoQ server. When the Start memoQ server link appears, click Start memoQ server.
- When the deployment tool reports that memoQ server is running again, close it.
Your programs may need files that cannot be stored in the project templates. These files can be stored in the project template - but if they are in the project template, you can't change or fix them while a project is running. In that case, you would need to create the whole project again.
Because of this, you need to place some - in fact, most - program files outside the project template.
- When you set up custom code execution, the service user will get the Log on locally system permission, as well as permissions for the C:\ProgramData\memoQ Server\Scripting folder on the server. Under this folder create another folder called Custom or Scripts, and place your program files there.
With this setup, if you need to replace a program file - because of a bugfix, for example -, you can do it in this folder. Your existing project will use the updated program file immediately.
Before you begin, you need at least two files:
- a main program file that does the actual transformation of the documents. It may need other files, too. You need to copy them with the main program file;
- a batch file that contains one single command: to start the main program file. This is the file that you put inside your project template.
If this program does not also transform the exported documents back, you need another program that helps with that. That means another two files at least.
If you run the scripts on your own computer, they will run in your name.
- Copy the main program file to a folder that you choose. Make sure you can run programs from that folder.
- Check the batch file, so that it points to the correct folder on the your computer. For example, the batch file can contain this command:
c:\Users\yourname\Documents\Scripting\C_Converter.exe %1 %2 %3 %4 %5 %6 %7 %8 %9
This batch file starts the program called C_Converter.exe, and passes up to nine arguments to it. The program is supposed to be in the C:\Users\yourname\Documents\Scripting folder, on the server. This is not a folder on your computer.
In fact, you can also use this command in your batch file - just replace the name of the .exe file if you're calling a different program.
- Return to memoQ, and edit the project template.
- Choose Automated actions, and click the Script before import tab.
- Under Settings for selected script, click Add files. Find and open the batch file that starts the main program.
To replace a script file or a batch file: Select the file in the Settings for selected scripts section, click Delete, and then add the new file by clicking Add files.
memoQ can use different scripts to pre-process different document types, or documents coming from different folders. To set one up, use the Script and file types section of the window.
- Type a file mask in the Wildcard expression matching the full path box. In the example about the XML Help files, you would type *.xml here. In Script name, type a descriptive name for this file type. For example: 'Help files with fragmented segments'. Click Add as new item.
- The file type and this name appears on the Script and input files list. Click that item.
- In the Settings for selected script section, select the batch file. If you have several batch files - because you plan to process different document types with different programs -, select the one that you want to use for this file type. Click Set as command.
- Set up the command next. The Command name is the same as the name of the batch file. In Command line arguments, you must include {InputFilePath} and {OutputFilePath}. These are automatically set by memoQ. The program - the script - must take an input file, and it must produce an output file. The input file is the source document as you receive it, and the output file is the transformed document that memoQ imports.
The program may require other command line arguments. For example, our XML transformer (that defragments XML Help files) requires this: -hm2xml {InputFilePath} {OutputFilePath} transform.log, where the file 'transform.log' is placed to the same folder where the main program is.
You can use other placeholders, too, so your program can work with the identifier of the project, as well as with the source and target languages. The program would mostly use these to find or to create folders so that different files for different purposes are kept apart.
To insert a placeholder: Click Insert placeholders, and choose the placeholder from the menu.
- InputFilePath: The full path and the name of the source document. You don't need to know what this path is - it's produced automatically by memoQ.
- OutputFilePath: The full path and the name of the transformed document that gets imported in the project. You don't need to know what this path is - it's produced automatically by memoQ.
- SrcLangIso2: Two-letter code of the source language of the project.
- SrcLangIso3: Three-letter code of the source language of the project.
- TrgLangListIso2: List of two-letter codes of the target languages of the project.
- TrgLangListIso3: List of three-letter codes of the target languages of the project.
- ProjectGuid: The globally unique identifier of the project. It's a long number, which is never repeated.
- Finally, you can test your script. Click Test run my program.
- The Custom code execution - test run window appears.
- Find a document that you need to transform anyway. Type or paste the document path and name in the text box, or click Browse to find the document on your computer. Click OK. memoQ attempts to run the script.
- When the test ends, the Test result message box appears, with an exit code. If the program was run successfully, the exit code is 0. If there is an error, the code is usually -1, or a different number.
After the pre-processing step is done, you need to set up the script after export. This will transform the exported document back into the format that your client expects.
- Click the Script after export tab. You need to set it up exactly the same way as the Script before import tab. Use the same file masks and descriptions under Script and input files.
- The difference is the command to run. Either there's a different batch file that runs on the exported file, or you use the same batch file, but with different command line arguments.
- If you use a different batch file: Add it using Add files. Select the file type in the list at the top. Select the new batch file, and then click Set as command.
- If you use the same batch file: Make sure it is selected for the same file type, and change the Command line arguments box, so that it performs the transformation back. For example, our script to export the XML Help files takes these arguments: -xml2hm {InputFilePath} {OutputFilePath} backform.log
- Use the Test run my command button to try your export script, too.
- After you finish, click OK.
Before you begin, you need at least two files:
- a main program file that does the actual transformation of the documents. It may need other files, too. You need to copy them with the main program file;
- a batch file that contains one single command: to start the main program file. This is the file that you put inside your project template.
If this program does not also transform the exported documents back, you need another program that helps with that. That means another two files at least.
- Copy the main program file to the server, in the Documents folder of the service user. (Example: c:\Users\_svc_mqsinternal\Documents\Scripting.) memoQ server will run the main program file under the name of the service user.
- Check the batch file, so that it points to the correct folder on the server. For example, the batch file can contain this command:
c:\Users\_svc_mqsinternal\Documents\Scripting\C_Converter.exe %1 %2 %3 %4 %5 %6 %7 %8 %9
This batch file starts the program called C_Converter.exe, and passes up to nine arguments to it. The program is supposed to be in the C:\Users\_svc_mqsinternal\Documents\Scripting folder, on the server. This is not a folder on your computer.
In fact, you can copy this command in the batch file - just replace the name of the .exe file if you're calling a different program. And check the folder, of course.
- Return to memoQ, connect to the same server, and edit the project template. The project template should be on the same server.
Online project, online template: Never use a local project to create online projects - make sure the template is on the same server.
- Choose Automated actions, and click the Script before import tab.
- Under Settings for selected script, click Add files. Find and open the batch file that starts the main program. Don't add the main program here.
To replace a script file or a batch file: Select the file in the Settings for selected scripts section, click Delete, and then add the new file by clicking Add files.
memoQ can use different scripts to pre-process different document types, or documents coming from different folders. To set one up, use the Script and file types section of the window.
- Type a file mask in the Wildcard expression matching the full path box. In the example about the XML Help files, you would type *.xml here. In Script name, type a descriptive name for this file type. For example: 'Help files with fragmented segments'. Click Add as new item.
- The file type and this name appears on the Script and input files list. Click that item.
- In the Settings for selected script section, select the batch file. If you have several batch files - because you plan to process different document types with different programs -, select the one that you want to use for this file type. Click Set as command.
- Set up the command next. The Command name is the same as the name of the batch file. In the Command line arguments, you must include {InputFilePath} and {OutputFilePath}. These are automatically set by memoQ. The program - the script - must take an input file, and it must produce an output file. The input file is the source document as you receive it, and the output file is the transformed document that memoQ imports.
The program may require other command line arguments, too. For example, our XML transformer (that defragments XML Help files) requires this: -hm2xml {InputFilePath} {OutputFilePath} transform.log, where the file 'transform.log' is placed in the same folder where the main program is.
You can use other placeholders, too, so your program can work with the identifier of the project, as well as with the source and target languages. The program would mostly use these to find or to create folders so that different files for different purposes are kept apart.
To insert a placeholder: Click Insert placeholders, and choose the placeholder from the menu.
- InputFilePath: The full path and the name of the source document. You don't need to know what this path is - it's produced automatically by memoQ.
- OutputFilePath: The full path and the name of the transformed document that gets imported in the project. You don't need to know what this path is - it's produced automatically by memoQ.
- SrcLangIso2: Two-letter code of the source language of the project.
- SrcLangIso3: Three-letter code of the source language of the project.
- TrgLangListIso2: List of two-letter codes of the target languages of the project.
- TrgLangListIso3: List of three-letter codes of the target languages of the project.
- ProjectGuid: The globally uniqiue identifier of the project. It's a long number, which is never repeated.
- Finally, you can test your script. Click Test run my program. The Custom code execution - test run window appears.
- Find a document that you need to transform anyway. Type or paste the document path and name in the text box, or click Browse to find the document on your computer. Click OK.
memoQ uploads the script and the sample document to the server, but not your external program. The memoQ attempts to run the script on the server.
- When the test ends, the Test result message box appears, with an exit code. If the program was run successfully, the exit code is 0. If there is an error, the code is usually -1, or a different number.
After the pre-processing step is done, you need to set up the script after export. This will transform the exported document back into the format that your client expects.
- Click the Script after export tab. You need to set it up exactly the same way as the Script before import tab. Use the same file masks and descriptions in Script and input files.
- The difference is the command to run. Either there's a different batch file that runs on the exported file, or you use the same batch file, but with different command line arguments.
- If you use a different batch file: Add it using Add files. Select the file type in the list at the top. Select the new batch file, and then click Set as command.
- If you use the same batch file: Make sure it is selected for the same file type, and change the Command line arguments box, so that it performs the transformation back. For example, our script to export the XML Help files takes these arguments: -xml2hm {InputFilePath} {OutputFilePath} backform.log
- Use the Test run my command button to try your export script, too.
- After you finish, click OK.
memoQ automatically runs the import and export scripts if the imported document matches the file type you set up.
For example, if you use scripts to process XML Help files, memoQ runs the scripts whenever you import a document with the .xml extension.
To use the scripts, create a project from the project template, and import a document with a name or extension that you set up in the script. The import script automatically runs before memoQ imports the document, and the export script automatically runs after memoQ exports the document.
You won't see the names of the transformed documents: In the Translations list, memoQ shows the original name and type of the imported document - not the name and type of the transformed document. As a result, the formatting and tags in the translation editor may slightly differ from what you would expect from the original file.
Normally, you would include your script - the entire program - in the project template. memoQ would extract them in a folder, and run them. This works well - until you discover that there is a problem with the script.
You may fix this in the project template. But when you change something in the template - replace the script, for example -, that will affect new projects only. If you have a problem in an existing project, that won't change when you replace the script in the template. If the entire program is in the template, you need to create the project again if you need to change the script.
The best practice is that you never put the main program logic in the project template. Instead, copy your program to a folder on the computer where the project will be - it's your own computer if it's a local project, and your memoQ server if it's an online project.
Then, add a simple batch file to the project template. The batch file has just one task: run the main program, and pass on the arguments (such as the file names) from the actual memoQ project. For an example, see Set up a script for an online project, earlier in this topic.
If there's a problem when you create a project from this project template, and you import documents, this is what you do:
- Fix the main program - or get it fixed by the developer you work with.
- Replace the program in the separate folder.
- Reimport the document(s). memoQ runs the same batch file, and it calls the main program - which changed, so the documents will be pre-processed differently this time.
You don't have to change the project template. What's more important, you don't have to re-create the project - simply reimport the documents, and see if the script was fixed.
When you finish
To save the project template, and return to Resource console: Click OK.
Possible next step: After you finish working on the template, close the Resource console, and create a project from the template.
To continue working on the project template: choose another category, and fine-tune other settings.
To return to Resource console, and not save the project template: Click Cancel.