No internet connection
  1. Home
  2. Support

Stuck on "crawling initial file list" with Local by Flywheel site

By tjc4 @tjc42020-03-11 16:21:41.749Z2020-03-11 16:34:49.225Z

Any tricks I need to know to generate a static site from a Local by Flywheel site?

I’ve got my local site running, I select the Manual / .zip deployment option, enter destination URL of http://mysite.com (where the local version is http://mysite.local), and click “start static site export.”

The export gets stuck almost immediately (see screenshot below).

I've disabled all browser extensions. Site is a pretty basic site built using Beaver Builder (plugin and theme). Local site is a copy of a normal, remotely hosted wordpress site. I've previously turned that remotely hosted site into a static site with no issues (albeit using Simply Static which isn't working on my local site either).

Any ideas?

UPDATE: It wasn't stuck. Just slow. After 20:51 it completed. But the resulting .zip is not valid. If I double click to see contents I see a blank folder (despite being 1.05 mb in size). If I try to extract, I get the error "windows cannot complete the extraction. the compressed (zipped) folder is invalid"

  • 8 replies
  1. Leon Stafford @leonstafford2020-03-11 18:19:20.456Z

    Hi @tjc4,

    I came to check which OS and see it's Windows. I haven't tested Local on Windows for a while. Could you please find the error logs for the webserver/PHP within Local and ideally send those, else just the 1MB zip to help@wp2static.com and I'll try to shed some more light on the issue.

    There's a new version of WP2Static soon ready for testing which has largely been rebuilt from the ground up and will solve many common issues. You can watch the GitHub repositories or join our public Telegram group to be informed of testable releases: https://t.me/wp2static

    Cheers,

    Leon

    1. Ttjc4 @tjc42020-03-11 19:03:29.978Z

      @leonstafford thanks for the reply and offer to help.

      I just sent you an email with the logs and .zip.

      Please let me know if you have any suggestions once you've had a chance to take a look.

      Thanks again!

      1. Leon Stafford @leonstafford2020-03-11 19:28:53.258Z

        Got it and replied, thanks.

        Stickout line for me is upstream prematurely closed connection while reading response header from upstream

        1. Leon Stafford @leonstafford2020-03-11 19:29:32.466Z

          Which usually denotes nginx.conf timeout settings being hit and we can increase those, reload/restart nginx and try again

      2. In reply toleonstafford:
        Ttjc4 @tjc42020-03-11 20:45:58.695Z

        @leonstafford thanks again for your help. I just completed the experiment described in my previous post.

        Experiment conditions:
        (1) \conf\nginx\site.conf.hbs > set fastcgi_read_timeout to 6000s
        (2) \conf\nginx\nginx.conf.hbs > set fastcgi_read_timeout to 6000s
        (3) set WP2Static Crawl Increment back to 1

        Experiment results:
        Process completed in 20:51 (mins:ss)
        .zip file was as before 1.05mb empty folder

        Interesting that changing timeout to 6000s seemed to have no impact on completion time.

        Any idea why the process completion time is so long?

        As mentioned previously, this local is a copy of a remotely hosted site. I just created a static site from that site. Took 20 seconds.

        And that site has more plugins activated (I disabled a few plugins on my local site to ensure they weren't the problem).

      3. T
        In reply totjc4:
        tjc4 @tjc42020-03-11 20:12:54.495Z

        Leon, thanks for the email. Posting my reply here for in case others have similar issues in the future...

        I didn't find nginx.conf but I did find nginx.conf.hbs in my site's \conf\nginx folder.

        That file has two timeouts: (1) keepalive_timeout  3; and (2) fastcgi_read_timeout 1800s;

        Note: the process doesn't fail.  It's completed 3x in times of 20:51, 20:53 and 20:47 (mm:ss).  See screenshot below.

        I'm happy to try changing the timeout values, let me know if you have an specific suggestions, but note that the observed times don't seem to align with the timeout values.

        Let's say average time is 20:50.  20x60+50 = 1250 seconds.  Doesn't seem close to 3 or 1800.

        I tried setting the Crawl Increment in WP2Static to Maximium (with no change to nginx timeouts) and attempted another static site export.

        With max crawl increment, I got a different outcome: Failed during "Crawling initial file list", please consult the Help tab for where to get assistance.

        UPDATE: I just found another conf file in the \conf\nginx folder: site.conf.hbs

        That file has one timeout: fastcgi_read_timeout 1200s;

        1200s aligns more closely with the observed times. I will try setting Crawl Increment back to 1 (since max seemed to cause issues) and changing the 1200s timeout to 6000s.

        I will report back with results from that experiment once available. In the interim, let me know if this new info provides any new insights.

        Thanks again!

        1. R@randomname2020-03-29 02:47:59.924Z

          I'm having the same issue. I'm using Local by Flywheel with default settings. Brand new Wordpress site with imported content from an old Wordpress site.

          Stuck on "Crawling initial file list".

          Error log shows a bunch of:

          2020/03/28 18:55:51 [error] 21068#11052: *1395 WSARecv() failed (10054: An existing connection was forcibly closed by the remote host) while reading response header from upstream, client: 127.0.0.1, server: , request: "POST /wp-admin/admin-ajax.php HTTP/1.1", upstream: "fastcgi://127.0.0.1:10004", host: "localhost:10000", referrer: "http://localhost:10000/wp-admin/admin.php?page=wp2static"

          1. Leon Stafford @leonstafford2020-03-31 19:31:21.705Z

            Hi @randomname, sorry for late reply. Version 7 pre-releases are working with Local as reported by a few users. If you submit your email on https://downloads.wp2static.com - you'll get a notification when the next pre-release is published (eta within next few days), which should help with this issue, also.