Skip to content

[pull] master from apify:master #242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pull merged 3 commits into from
Jul 15, 2026
Merged

[pull] master from apify:master #242

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2c4eef2
chore(deps): update actions/setup-node action to v7 (#3854)
renovate[bot] Jul 15, 2026
b3c3ef2
fix(core): skip malformed cookie fragments in mergeCookies (#3822)
anxkhn Jul 15, 2026
3e3dc63
ci: fix E2E tests (#3855)
B4nan Jul 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- uses: actions/checkout@v7

- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: 24
package-manager-cache: false
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish-to-npm.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
fetch-depth: 0

- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: 24
package-manager-cache: false
Expand All @@ -55,7 +55,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
uses: actions/checkout@v7

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: ${{ matrix.node-version }}
package-manager-cache: false
Expand All @@ -57,7 +57,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down Expand Up @@ -105,7 +105,7 @@ jobs:
fetch-depth: 0

- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: 24
package-manager-cache: false
Expand All @@ -116,7 +116,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down Expand Up @@ -200,7 +200,7 @@ jobs:
fetch-depth: 0

- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: 24
package-manager-cache: false
Expand All @@ -214,7 +214,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down
16 changes: 8 additions & 8 deletions .github/workflows/test-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
uses: actions/checkout@v7

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: ${{ matrix.node-version }}
package-manager-cache: false
Expand All @@ -45,7 +45,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down Expand Up @@ -109,7 +109,7 @@ jobs:
uses: actions/checkout@v7

- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: 24
package-manager-cache: false
Expand All @@ -120,7 +120,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down Expand Up @@ -251,7 +251,7 @@ jobs:
uses: actions/checkout@v7

- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: 24
package-manager-cache: false
Expand All @@ -262,7 +262,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down Expand Up @@ -298,7 +298,7 @@ jobs:
fetch-depth: 0

- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: 24
package-manager-cache: false
Expand All @@ -309,7 +309,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down
6 changes: 4 additions & 2 deletions .github/workflows/test-e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
uses: actions/checkout@v7

- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
node-version: 24
package-manager-cache: false
Expand All @@ -40,7 +40,7 @@ jobs:
corepack prepare yarn@stable --activate

- name: Activate cache for Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v7
with:
cache: 'yarn'

Expand Down Expand Up @@ -87,6 +87,8 @@ jobs:
run: yarn test:e2e
env:
STORAGE_IMPLEMENTATION: ${{ matrix.storage }}
# Provided explicitly: the CLI may store the token in the OS keyring, where the tests can't read it.
APIFY_TOKEN: ${{ secrets.APIFY_SCRAPER_TESTS_API_TOKEN }}
APIFY_HTTPBIN_TOKEN: ${{ secrets.APIFY_HTTPBIN_TOKEN }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GITHUB_TOKEN: ${{ github.token }}
3 changes: 2 additions & 1 deletion packages/core/src/cookie_utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,8 @@ export function mergeCookies(url: string, sourceCookies: string[]): string {
// ignore extra spaces
if (!cookieString) continue;

const cookie = Cookie.parse(cookieString)!;
const cookie = Cookie.parse(cookieString);
if (!cookie) throw new CookieParseError(cookieString);
const similarKeyCookie = jar.getCookiesSync(url).find((c) => {
return cookie.key !== c.key && cookie.key.toLowerCase() === c.key.toLowerCase();
});
Expand Down
9 changes: 9 additions & 0 deletions test/core/crawlers/cheerio_crawler.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1169,6 +1169,15 @@ describe('CheerioCrawler', () => {
);
});

test('mergeCookies() throws a contextual error for malformed cookie fragments', () => {
expect(() => mergeCookies('https://example.com', ['valid=1; brokenfragment'])).toThrow(
'Could not parse cookie header string: brokenfragment',
);
expect(() => mergeCookies('https://example.com', ['sessionid'])).toThrow(
'Could not parse cookie header string: sessionid',
);
});

test('should use sessionId in proxyUrl when the session pool is enabled', async () => {
const sourcesNew = [{ url: 'http://example.com/?q=1' }];
const requestListNew = await RequestList.open({ sources: sourcesNew });
Expand Down
2 changes: 2 additions & 0 deletions test/e2e/adaptive-playwright-robots-file/actor/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ await Actor.init({
});

const crawler = new AdaptivePlaywrightCrawler({
// The store rate-limits the platform's shared egress IP, so crawl through a proxy.
proxyConfiguration: await Actor.createProxyConfiguration(),
maxRequestsPerCrawl: 10,
respectRobotsTxtFile: true,
onSkippedRequest: (args) => crawler.log.warningOnce(`Request ${args.url} was skipped, reason: ${args.reason}`),
Expand Down
2 changes: 2 additions & 0 deletions test/e2e/cheerio-robots-file/actor/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ await Actor.init({
});

const crawler = new CheerioCrawler({
// The store rate-limits the platform's shared egress IP, so crawl through a proxy.
proxyConfiguration: await Actor.createProxyConfiguration(),
maxRequestsPerCrawl: 10,
respectRobotsTxtFile: true,
});
Expand Down
2 changes: 2 additions & 0 deletions test/e2e/playwright-introduction-guide/actor/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@ router.addDefaultHandler(async ({ request, page, enqueueLinks, log }) => {
});

const crawler = new PlaywrightCrawler({
// The store rate-limits the platform's shared egress IP, so crawl through a proxy.
proxyConfiguration: await Actor.createProxyConfiguration(),
maxRequestsPerCrawl: 15, // so the test runs faster
// Instead of the long requestHandler with
// if clauses we provide a router instance.
Expand Down
2 changes: 2 additions & 0 deletions test/e2e/playwright-robots-file/actor/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ await Actor.init({
});

const crawler = new PlaywrightCrawler({
// The store rate-limits the platform's shared egress IP, so crawl through a proxy.
proxyConfiguration: await Actor.createProxyConfiguration(),
maxRequestsPerCrawl: 10,
respectRobotsTxtFile: true,
});
Expand Down
2 changes: 2 additions & 0 deletions test/e2e/puppeteer-store-pagination-jquery/actor/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ const mainOptions = {

await Actor.main(async () => {
const crawler = new PuppeteerCrawler({
// The store rate-limits the platform's shared egress IP, so crawl through a proxy.
proxyConfiguration: await Actor.createProxyConfiguration(),
maxRequestsPerCrawl: 10,
preNavigationHooks: [
async ({ page }, goToOptions) => {
Expand Down
2 changes: 2 additions & 0 deletions test/e2e/puppeteer-store-pagination/actor/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ await Actor.init({
});

const crawler = new PuppeteerCrawler({
// The store rate-limits the platform's shared egress IP, so crawl through a proxy.
proxyConfiguration: await Actor.createProxyConfiguration(),
maxRequestsPerCrawl: 10,
preNavigationHooks: [
async ({ page }, goToOptions) => {
Expand Down
9 changes: 9 additions & 0 deletions test/e2e/tools.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,15 @@ export async function getApifyToken() {
}

const { token } = await fs.readJSON(authPath);

// Newer CLI versions keep the token in the OS keyring, leaving auth.json without it.
if (!token) {
throw new Error(
'Your Apify token is not stored in auth.json (the CLI likely saved it in the OS keyring). ' +
'Set the "APIFY_TOKEN" environment variable, or re-run "apify login" with "APIFY_DISABLE_KEYRING=1".',
);
}

return token;
}

Expand Down
Loading