Welcome to WP Content Crawler’s documentation
- 1. Getting Started
- 2. Quick Start
- 3. Guides
- 3.1. Creating site settings to crawl a site
- 3.2. Saving posts automatically
- 3.3. Updating posts automatically
- 3.4. Translating posts automatically
- 3.5. Spinning posts automatically
- 3.6. Deleting posts automatically
- 3.7. Using cookies
- 3.8. Increasing speed of tests in site settings page
- 3.9. Using custom general settings for a site
- 3.10. Adding category URLs automatically
- 3.11. Duplicating site settings
- 3.12. Manually saving posts
- 3.13. Manually updating posts
- 3.14. Tracking saved posts
- 3.15. Getting notified when selectors cannot find a value
- 3.16. Testing site settings
- 3.17. Collecting post URLs from multiple pages of a category
- 3.18. Saving multi-page posts
- 3.19. Saving list-type posts
- 3.20. Saving post meta (custom fields)
- 3.21. Saving taxonomy values
- 3.22. Creating post categories automatically
- 3.23. Saving posts as custom post types
- 3.24. Disabling fixed navigation and tabs in site settings page
- 3.25. Saving post title
- 3.26. Saving post excerpt
- 3.27. Saving post content
- 3.28. Saving post tags
- 3.29. Saving post permalink (slug)
- 3.30. Saving post date
- 3.31. Saving the featured image of a post
- 3.32. Saving images in post content
- 3.33. Saving lazy-loaded images
- 3.34. Defining custom short codes to use anything in templates
- 3.35. Saving WooCommerce products
- 3.36. Saving featured images from category pages
- 3.37. Defining how posts are marked as duplicate
- 3.38. Adding/removing things to/from post content
- 3.39. Adding/removing things to/from post title
- 3.40. Adding/removing things to/from post excerpt
- 3.41. Saving posts as draft or pending (defining post status)
- 3.42. Setting the author of the posts
- 3.43. Removing the links in post content
- 3.44. Showing
iframe
s in post content - 3.45. Showing
script
s in post content - 3.46. Creating a gallery of images or files in post content
- 3.47. Taking notes about site settings
- 3.48. Changing something in every page of every site
- 3.49. Removing an HTML element from post content
- 3.50. Dealing with character encoding problems
- 3.51. Using proxies
- 3.52. Adding/removing/changing things in target page
- 3.53. Limiting maximum tags posts can have
- 3.54. Setting password to posts
- 3.55. Setting HTTP User Agent and HTTP Accept values
- 3.56. Manually adding post URLs
- 3.57. Deleting post URLs of a site
- 4. Video Tutorials
- 5. Site Listing Page
- 6. Site Settings Page
- 6.1. Creating a New Site
- 6.2. Saving The Settings
- 6.3. Tabs
- 6.3.1. Main Tab
- 6.3.1.1. Main
- 6.3.1.1.1. Site URL
- 6.3.1.1.2. Active for scheduling?
- 6.3.1.1.3. Active for recrawling?
- 6.3.1.1.4. Active for post deleting?
- 6.3.1.1.5. Active for post translation?
- 6.3.1.1.6. Translatable Fields
- 6.3.1.1.7. Active for post spinning?
- 6.3.1.1.8. Spinnable Fields
- 6.3.1.1.9. Check duplicate posts via
- 6.3.1.1.10. Use custom general settings?
- 6.3.1.1.11. Cookies
- 6.3.1.2. Settings Page Section
- 6.3.1.1. Main
- 6.3.2. Category Tab
- 6.3.3. Post Tab
- 6.3.3.1. Post
- 6.3.3.2. Category Section
- 6.3.3.3. WooCommerce Section
- 6.3.3.4. Date Section
- 6.3.3.5. Meta Section
- 6.3.3.6. Featured Image Section
- 6.3.3.7. Images Section
- 6.3.3.7.1. Save all images in the post content?
- 6.3.3.7.2. Save images as media?
- 6.3.3.7.3. Save images as gallery?
- 6.3.3.7.4. Gallery Image URL Selectors
- 6.3.3.7.5. Save images as WooCommerce product gallery?
- 6.3.3.7.6. Image URL Selectors
- 6.3.3.7.7. Test Image URL
- 6.3.3.7.8. Find and replace in image URLs
- 6.3.3.8. Custom Short Codes Section
- 6.3.3.9. List Type Posts Section
- 6.3.3.10. Pagination Section
- 6.3.3.11. Post Meta Section
- 6.3.3.12. Taxonomies Section
- 6.3.3.13. Manipulate HTML Section
- 6.3.3.14. Unnecessary Elements Section
- 6.3.3.15. Notifications Section
- 6.3.4. Templates Tab
- 6.3.4.1. Templates
- 6.3.4.2. Quick Fixes Section
- 6.3.4.3. Unnecessary Elements Section
- 6.3.4.4. Manipulate HTML Section
- 6.3.4.4.1. Find and Replace Test Code
- 6.3.4.4.2. Find and replace in post’s content
- 6.3.4.4.3. Find and replace in custom short code contents
- 6.3.4.4.4. Find and replace in post’s title
- 6.3.4.4.5. Find and replace in post’s excerpt
- 6.3.4.4.6. Find and replace in post’s each tag
- 6.3.4.4.7. Find and replace in meta keywords
- 6.3.4.4.8. Find and replace in meta description
- 6.3.5. Notes Tab
- 6.3.6. Settings Tab
- 6.3.7. Import/Export Tab
- 6.3.1. Main Tab
- 6.4. Visual Inspector
- 6.4.1. Back
- 6.4.2. Forward
- 6.4.3. Refresh
- 6.4.4. Address bar
- 6.4.5. Go
- 6.4.6. Show sidebar
- 6.4.7. Use CSS selector
- 6.4.8. CSS selector
- 6.4.9. Test CSS selector
- 6.4.10. Clear highlights
- 6.4.11. Show alternative CSS selectors
- 6.4.12. Remove elements
- 6.4.13. Toggle hover select
- 6.4.14. Target HTML tag
- 6.4.15. Selection behavior
- 6.4.16. Test type
- 6.4.17. Manipulations
- 6.4.18. Use immediately
- 6.4.19. Remove scripts
- 6.4.20. Remove styles
- 6.4.21. URL History
- 6.4.22. Delete URL history
- 6.4.23. Alternative CSS selectors
- 6.4.24. All used selectors
- 6.4.25. Show/hide section
- 6.4.26. Close sidebar
- 6.5. Lifecycle of Events
- 7. General Settings Page
- 7.1. Scheduling Tab
- 7.1.1. Scheduling
- 7.1.1.1. Scheduling is active?
- 7.1.1.2. Post URL Collection Interval
- 7.1.1.3. Post Crawl Interval
- 7.1.1.4. Maximum number of pages to crawl per category
- 7.1.1.5. Maximum number of pages to check to find new post URLs
- 7.1.1.6. Run count for URL collection event
- 7.1.1.7. Run count for post crawling event
- 7.1.2. Recrawling Section
- 7.1.3. Deleting Section
- 7.1.1. Scheduling
- 7.2. Post Tab
- 7.3. Translation Tab
- 7.4. Spinning Tab
- 7.4.1. Spinning
- 7.4.2. Spin Rewriter API Options
- 7.4.2.1. Email address
- 7.4.2.2. API Key
- 7.4.2.3. Confidence Level
- 7.4.2.4. Auto Protected Terms
- 7.4.2.5. Nested Spintax
- 7.4.2.6. Auto Sentences
- 7.4.2.7. Auto Paragraphs
- 7.4.2.8. Auto New Paragraphs
- 7.4.2.9. Auto Sentence Trees
- 7.4.2.10. Use Only Synonyms
- 7.4.2.11. Reorder Paragraphs
- 7.4.2.12. Text with spintax?
- 7.4.2.13. Test Spinning Options
- 7.4.2.14. Check API Usage Statistics
- 7.4.3. Türkçe Spin API Options
- 7.5. SEO Tab
- 7.6. Notifications Tab
- 7.7. Advanced Tab
- 7.1. Scheduling Tab
- 8. Tester Page
- 9. Dashboard Page
- 10. Tools Page
- 11. License Settings Page
- 12. Common Options
- 12.1. Selector and Attribute Setting
- 12.2. Find and Replace Setting
- 12.2.1. Examples
- 12.2.1.1. Find and replace a word in a sentence in a case-sensitive way
- 12.2.1.2. Remove a word from a sentence
- 12.2.1.3. Find and replace a word in a sentence in a case-insensitive way
- 12.2.1.4. Find and replace by using a regular expression
- 12.2.1.5. Find and use something from the found text in the replacement
- 12.2.1.6. Replace
data-src
attribute’s value withsrc
attribute’s value
- 12.2.1. Examples
- 12.3. Options Box
- 12.4. Manipulate HTML Section
- 12.5. Unnecessary Elements Section
- 12.6. Transformable Fields
- 13. Permissions
- 14. Frequently Asked Questions
- 14.1. Where is my license key?
- 14.2. Can I get content from X site?
- 14.3. Can I save custom fields/taxonomies of a plugin/theme?
- 14.4. How to change the domain registered to my license
- 14.5. In how many domains can I use my license?
- 14.6. I cannot crawl sites that use SSL (https)
- 14.7.
embed
,script
, andiframe
codes are not shown properly, or not shown at all - 14.8. How to change PHP version and enable
mbstring
extension? - 14.9. How to remove links from images
- 14.10. Automatic crawling works only when someone browses my site
- 14.11. I installed the plugin on a test site and I want to use the plugin with the same license code on another site. Can I do it?
- 14.12. How to send a support request
- 14.13. What do I do when admin and FTP login credentials are asked from me?
- 14.14. The plugin does not work
- 15. Troubleshooting
- 15.1. Checking if CRON jobs work and fixing CRON problems
- 15.2. Fixing error messages like “Parse error: syntax error…”
- 15.3. Fixing automatic crawling problems
- 15.4. Fixing invisible tab contents in setting pages
- 15.5. Fixing unsaved settings problem
- 15.6. Fixing empty plugin pages problem
- 15.7. Fixing the problem of empty pages shown after saving the settings
- 15.8. Fixing the problem of error pages shown after saving the settings
- 15.9. Fixing automatic crawling problems caused by the theme Newspaper 8 (or later)
- 15.10. Fixing cURL-related automatic crawling problems
- 15.11. Fixing duplicate post issues
- 15.12. Fixing character encoding problems
- 15.13. Manual Crawling Tool’s buttons do not work
- 15.14. Troubleshooting proxy issues
- 15.15. Fixing the problems caused by other plugins/themes
- 15.16. Debugging errors
- 16. Changelog
- 16.1. v1.10.2 - 31 January 2021
- 16.2. v1.10.1 - 25 August 2020
- 16.3. v1.10.0 - 13 February 2020
- 16.4. v1.9.0 - 13 December 2019
- 16.5. v1.8.0 - 1 January 2019
- 16.6. v1.7.0 - 22 October 2017
- 16.7. v1.6.0 - 4 March 2017
- 16.8. v1.5.1 - 7 February 2017
- 16.9. v1.4.1 - 27 January 2017
- 16.10. v1.4.0 - 26 January 2017
- 16.11. v1.3.0 - 14 January 2017
- 16.12. v1.2.0 - 30 August 2016