Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-1502

Test for CrawlDatum state transitions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.7, 2.2
    • 1.9
    • crawldb
    • None
    • Patch Available

    Description

      An exhaustive test to check the matrix of CrawlDatum state transitions (CrawlStatus in 2.x) would be useful to detect errors esp. for continuous crawls where the number of possible transitions is quite large. Additional factors with impact on state transitions (retry counters, static and dynamic intervals) are also tested.
      The tests will help to address the NUTCH-578 and NUTCH-1245. See the latter for a first sketchy patch.

      Attachments

        1. NUTCH-1502-trunk-v1.patch
          43 kB
          Sebastian Nagel

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. NUTCH-1564 AdaptiveFetchSchedule: sync_delta forces immediate refetch for documents not modified

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. NUTCH-1422 bypass signature comparison when a document is redirected

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              Unassigned Unassigned
              snagel Sebastian Nagel
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: