Welcome to WP Content Crawler’s documentation
- 1. Getting Started
- 2. Quick Start
- 3. Guides
- 3.1. Creating site settings to crawl a site
- 3.2. Saving posts automatically
- 3.3. Updating posts automatically
- 3.4. Translating posts automatically
- 3.5. Spinning posts automatically
- 3.6. Deleting posts automatically
- 3.7. Using cookies
- 3.8. Increasing speed of tests in site settings page
- 3.9. Using custom general settings for a site
- 3.10. Adding category URLs automatically
- 3.11. Duplicating site settings
- 3.12. Manually saving posts
- 3.13. Manually updating posts
- 3.14. Tracking saved posts
- 3.15. Getting notified when selectors cannot find a value
- 3.16. Testing site settings
- 3.17. Collecting post URLs from multiple pages of a category
- 3.18. Saving multi-page posts
- 3.19. Saving list-type posts
- 3.20. Saving post meta (custom fields)
- 3.21. Saving taxonomy values
- 3.22. Creating post categories automatically
- 3.23. Saving posts as custom post types
- 3.24. Disabling fixed navigation and tabs in site settings page
- 3.25. Saving post title
- 3.26. Saving post excerpt
- 3.27. Saving post content
- 3.28. Saving post tags
- 3.29. Saving post permalink (slug)
- 3.30. Saving post date
- 3.31. Saving the featured image of a post
- 3.32. Saving images in post content
- 3.33. Saving lazy-loaded images
- 3.34. Defining custom short codes to use anything in templates
- 3.35. Saving WooCommerce products
- 3.36. Saving featured images from category pages
- 3.37. Defining how posts are marked as duplicate
- 3.38. Adding/removing things to/from post content
- 3.39. Adding/removing things to/from post title
- 3.40. Adding/removing things to/from post excerpt
- 3.41. Saving posts as draft or pending (defining post status)
- 3.42. Setting the author of the posts
- 3.43. Removing the links in post content
- 3.44. Showing
iframe
s in post content - 3.45. Showing
script
s in post content - 3.46. Creating a gallery of images or files in post content
- 3.47. Taking notes about site settings
- 3.48. Changing something in every page of every site
- 3.49. Removing an HTML element from post content
- 3.50. Dealing with character encoding problems
- 3.51. Using proxies
- 3.52. Adding/removing/changing things in target page
- 3.53. Limiting maximum tags posts can have
- 3.54. Setting password to posts
- 3.55. Setting HTTP User Agent and HTTP Accept values
- 3.56. Manually adding post URLs
- 3.57. Deleting post URLs of a site
- 4. Site Listing Page
- 5. Site Settings Page
- 5.1. Creating a New Site
- 5.2. Saving The Settings
- 5.3. Tabs
- 5.3.1. Main Tab
- 5.3.1.1. Main
- 5.3.1.1.1. Site URL
- 5.3.1.1.2. Active for scheduling?
- 5.3.1.1.3. Active for recrawling?
- 5.3.1.1.4. Active for post deleting?
- 5.3.1.1.5. Active for post translation?
- 5.3.1.1.6. Translatable Fields
- 5.3.1.1.7. Active for post spinning?
- 5.3.1.1.8. Spinnable Fields
- 5.3.1.1.9. Check duplicate posts via
- 5.3.1.1.10. Use custom general settings?
- 5.3.1.1.11. Cookies
- 5.3.1.2. Settings Page Section
- 5.3.1.1. Main
- 5.3.2. Category Tab
- 5.3.3. Post Tab
- 5.3.3.1. Post
- 5.3.3.2. Category Section
- 5.3.3.3. WooCommerce Section
- 5.3.3.4. Date Section
- 5.3.3.5. Meta Section
- 5.3.3.6. Featured Image Section
- 5.3.3.7. Images Section
- 5.3.3.7.1. Save all images in the post content?
- 5.3.3.7.2. Save images as media?
- 5.3.3.7.3. Save images as gallery?
- 5.3.3.7.4. Gallery Image URL Selectors
- 5.3.3.7.5. Save images as WooCommerce product gallery?
- 5.3.3.7.6. Image URL Selectors
- 5.3.3.7.7. Test Image URL
- 5.3.3.7.8. Find and replace in image URLs
- 5.3.3.8. Custom Short Codes Section
- 5.3.3.9. List Type Posts Section
- 5.3.3.10. Pagination Section
- 5.3.3.11. Post Meta Section
- 5.3.3.12. Taxonomies Section
- 5.3.3.13. Manipulate HTML Section
- 5.3.3.14. Unnecessary Elements Section
- 5.3.3.15. Notifications Section
- 5.3.4. Templates Tab
- 5.3.4.1. Templates
- 5.3.4.2. Quick Fixes Section
- 5.3.4.3. Unnecessary Elements Section
- 5.3.4.4. Manipulate HTML Section
- 5.3.4.4.1. Find and Replace Test Code
- 5.3.4.4.2. Find and replace in post’s content
- 5.3.4.4.3. Find and replace in custom short code contents
- 5.3.4.4.4. Find and replace in post’s title
- 5.3.4.4.5. Find and replace in post’s excerpt
- 5.3.4.4.6. Find and replace in post’s each tag
- 5.3.4.4.7. Find and replace in meta keywords
- 5.3.4.4.8. Find and replace in meta description
- 5.3.5. Notes Tab
- 5.3.6. Settings Tab
- 5.3.7. Import/Export Tab
- 5.3.1. Main Tab
- 5.4. Visual Inspector
- 5.4.1. Back
- 5.4.2. Forward
- 5.4.3. Refresh
- 5.4.4. Address bar
- 5.4.5. Go
- 5.4.6. Show sidebar
- 5.4.7. Use CSS selector
- 5.4.8. CSS selector
- 5.4.9. Test CSS selector
- 5.4.10. Clear highlights
- 5.4.11. Show alternative CSS selectors
- 5.4.12. Remove elements
- 5.4.13. Toggle hover select
- 5.4.14. Target HTML tag
- 5.4.15. Selection behavior
- 5.4.16. Test type
- 5.4.17. Manipulations
- 5.4.18. Use immediately
- 5.4.19. Remove scripts
- 5.4.20. Remove styles
- 5.4.21. URL History
- 5.4.22. Delete URL history
- 5.4.23. Alternative CSS selectors
- 5.4.24. All used selectors
- 5.4.25. Show/hide section
- 5.4.26. Close sidebar
- 5.5. Lifecycle of Events
- 6. General Settings Page
- 6.1. Scheduling Tab
- 6.1.1. Scheduling
- 6.1.1.1. Scheduling is active?
- 6.1.1.2. Post URL Collection Interval
- 6.1.1.3. Post Crawl Interval
- 6.1.1.4. Maximum number of pages to crawl per category
- 6.1.1.5. Maximum number of pages to check to find new post URLs
- 6.1.1.6. Run count for URL collection event
- 6.1.1.7. Run count for post crawling event
- 6.1.2. Recrawling Section
- 6.1.3. Deleting Section
- 6.1.1. Scheduling
- 6.2. Post Tab
- 6.3. Translation Tab
- 6.4. Spinning Tab
- 6.4.1. Spinning
- 6.4.2. Spin Rewriter API Options
- 6.4.2.1. Email address
- 6.4.2.2. API Key
- 6.4.2.3. Confidence Level
- 6.4.2.4. Auto Protected Terms
- 6.4.2.5. Nested Spintax
- 6.4.2.6. Auto Sentences
- 6.4.2.7. Auto Paragraphs
- 6.4.2.8. Auto New Paragraphs
- 6.4.2.9. Auto Sentence Trees
- 6.4.2.10. Use Only Synonyms
- 6.4.2.11. Reorder Paragraphs
- 6.4.2.12. Text with spintax?
- 6.4.2.13. Test Spinning Options
- 6.4.2.14. Check API Usage Statistics
- 6.5. SEO Tab
- 6.6. Notifications Tab
- 6.7. Advanced Tab
- 6.1. Scheduling Tab
- 7. Tester Page
- 8. Dashboard Page
- 9. Tools Page
- 10. License Settings Page
- 11. Common Options
- 11.1. Selector and Attribute Setting
- 11.2. Find and Replace Setting
- 11.2.1. Examples
- 11.2.1.1. Find and replace a word in a sentence in a case-sensitive way
- 11.2.1.2. Remove a word from a sentence
- 11.2.1.3. Find and replace a word in a sentence in a case-insensitive way
- 11.2.1.4. Find and replace using a regular expression
- 11.2.1.5. Find and use something from the found text in the replacement
- 11.2.1.6. Replace
data-src
attribute’s value withsrc
attribute’s value
- 11.2.1. Examples
- 11.3. Options Box
- 11.4. Manipulate HTML Section
- 11.5. Unnecessary Elements Section
- 11.6. Transformable Fields
- 12. Permissions
- 13. Frequently Asked Questions
- 13.1. Where is my license key?
- 13.2. Can I get content from X site?
- 13.3. Can I save custom fields/taxonomies of a plugin/theme?
- 13.4. How to change the domain registered to my license
- 13.5. In how many domains can I use my license?
- 13.6. I cannot crawl sites using SSL (https)
- 13.7.
embed
,script
, andiframe
codes are not shown properly, or not shown at all - 13.8. How to change PHP version and enable
mbstring
extension? - 13.9. How to remove links from images
- 13.10. Automatic crawling works only when someone browses my site
- 13.11. I installed the plugin on a test site and I want to use the plugin with the same license code on another site. Can I do it?
- 13.12. How to send a support request
- 13.13. What do I do when admin and FTP login credentials are asked from me?
- 13.14. The plugin does not work
- 14. Troubleshooting
- 14.1. Checking if CRON jobs work and fixing CRON problems
- 14.2. Fixing error messages like “Parse error: syntax error…”
- 14.3. Fixing automatic crawling problems
- 14.4. Fixing invisible tab contents in setting pages
- 14.5. Fixing unsaved settings problem
- 14.6. Fixing empty plugin pages problem
- 14.7. Fixing the problem of empty pages shown after saving the settings
- 14.8. Fixing the problem of error pages shown after saving the settings
- 14.9. Fixing automatic crawling problems caused by the theme Newspaper 8 (or later)
- 14.10. Fixing cURL-related automatic crawling problems
- 14.11. Fixing duplicate post issues
- 14.12. Fixing character encoding problems
- 14.13. Manual Crawling Tool’s buttons do not work
- 14.14. Fixing the problems caused by other plugins/themes
- 14.15. Debugging errors
- 15. Changelog