When the basic approach of submitting a form using request-promise isn't enough (e.g. it is hard to correctly replicate the request), we can use Puppeteer to simulate submitting the same way a human-operated browser would.
Downloading the file
After creating a new actor, the first thing necessary is to download the file - we can do that using the request-promise module. We will also be using the fs module to save it to the hard drive, so make sure they are included.
const fs = require('fs');
const request = require('request-promise');The actual downloading is going to be slightly different for text and binary files. For a text file, it can simply be done like this:
const fileData = await request('https://some-site.com/file.txt');For a binary file, we need to provide an additional parameter so as not to interpret it as text:
const fileData = await request({
uri: 'https://some-site.com/file.pdf',
encoding: null
});In this case, fileData will be a Buffer instead of String.
To use the file in Puppeteer, we need to save it to the hard drive. This can be done using the fs package.
fs.writeFileSync('./file.pdf', fileData);Submitting the form
The first step necessary is to open the form page in Puppeteer. This can be done as follows:
const browser = await Apify.launchPuppeteer();
const page = await browser.newPage();
await page.goto('https://some-site.com/file-upload.php');To fill in any necessary form inputs, we can use the page.type function. This works even in cases when elem.value = 'value' is not usable.
await page.type('input[name=firstName]', 'John');
await page.type('input[name=surname]', 'Doe');
await page.type('input[name=email]', 'john.doe@mail.com');To add the file to the appropriate input, we first need to find it and then use the uploadFile function.
const fileInput = await page.$('input[type=file]');
await fileInput.uploadFile('./file.pdf');Now we can finally submit the form.
await page.evaluate(() => {
document.querySelector('input[type=submit]').click();
});