[wget-notify] [bug #23306] -N doesn't work on robots.txt

[Micah Cowan]

URL:
  <http://savannah.gnu.org/bugs/?23306>

                 Summary: -N doesn't work on robots.txt
                 Project: GNU Wget
            Submitted by: micahcowan
            Submitted on: Tuesday 05/20/2008 at 15:28
                Category: Program Logic
                Severity: 2 - Minor
                Priority: 5 - Normal
                  Status: Confirmed
                 Privacy: Public
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: None
        Operating System: None
         Reproducibility: None
           Fixed Release: None
         Planned Release: 1.13
              Regression: None
           Work Required: 0 - Hours
          Patch Included: None

    _______________________________________________________

Details:

At least, in combination with -nd, -nH, etc.:


wget -N -p -nd -nH micah.cowan.name
--2008-05-20 15:03:30--  http://micah.cowan.name/
Resolving micah.cowan.name... 66.150.225.51
Connecting to micah.cowan.name|66.150.225.51|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Last-modified header missing -- time-stamps turned off.
--2008-05-20 15:03:30--  http://micah.cowan.name/
Connecting to micah.cowan.name|66.150.225.51|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `index.html'

    [  <=>                                  ] 69,284       171K/s   in 0.4s  
 

2008-05-20 15:03:31 (171 KB/s) - `index.html' saved [69284]

Loading robots.txt; please ignore errors.
--2008-05-20 15:03:31--  http://micah.cowan.name/robots.txt
Connecting to micah.cowan.name|66.150.225.51|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 358 [text/plain]
Saving to: `robots.txt.2'

100%[======================================>] 358         --.-K/s   in 0s    
 

2008-05-20 15:03:31 (2.30 MB/s) - `robots.txt.2' saved [358/358]

--2008-05-20 15:03:31-- 
http://micah.cowan.name/wp-content/themes/ocadia/style.css
Reusing existing connection to micah.cowan.name:80.
HTTP request sent, awaiting response... 200 OK
Length: 7287 (7.1K) [text/css]
Server file no newer than local file `style.css' -- not retrieving.

--2008-05-20 15:03:31-- 
http://micah.cowan.name/wp-includes/images/smilies/icon_wink.gif
Reusing existing connection to micah.cowan.name:80.
HTTP request sent, awaiting response... 200 OK
Length: 170 [image/gif]
Server file no newer than local file `icon_wink.gif' -- not retrieving.

--2008-05-20 15:03:32-- 
http://micah.cowan.name/wp-includes/images/smilies/icon_smile.gif
Reusing existing connection to micah.cowan.name:80.
HTTP request sent, awaiting response... 200 OK
Length: 174 [image/gif]
Server file no newer than local file `icon_smile.gif' -- not retrieving.

FINISHED --2008-05-20 15:03:32--
Downloaded: 2 files, 68K in 0.4s (172 KB/s)





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?23306>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/