Search the html source of every page on a website for a specific string

Python script

import requests

def search_string_in_pages(url, search_string):
    try:
        response = requests.get(url)
        if response.status_code == 200:
            html_content = response.text
            if search_string in html_content:
                print("Found '{}' in: {}".format(search_string, url))
        else:
            print("Failed to retrieve page: {}".format(url))
    except Exception as e:
        print("An error occurred:", e)

# List of URLs to search
urls = ["http://example.com/page1", "http://example.com/page2", ...]

# String to search for
search_string = "your specific string"

for url in urls:
    search_string_in_pages(url, search_string)

How to run a Python script

Running a Python script on your Mac is straightforward. You can follow these steps:

1. Open Terminal: You can find Terminal in the Utilities folder within the Applications folder, or you can search for it using Spotlight (Cmd + Space, then type “Terminal”).

2. Navigate to the Directory Containing Your Script: Use the cd command to navigate to the directory where your Python script is located. For example, if your script is in the Documents folder, you can navigate there with:

cd ~/Documents

1. Run the Script: Once you’re in the correct directory, you can run your Python script by typing:

python your_script.py

Your Python script will then run, and you’ll see any output or errors displayed in the Terminal window.

If you’re using a specific Python environment or version, such as a virtual environment or Python 3, you might need to adjust the python command accordingly. For example, if you’re using Python 3, you would use python3 instead of python.

How to fix Python Not Found error

If you’re encountering a “command not found: python” error, it likely means that Python isn’t installed on your system or isn’t included in your system’s PATH variable. Here’s how you can fix it:

1. Check if Python is Installed: Open Terminal and type the following command:

python --version

If Python is installed, this command will display the installed Python version. If it’s not installed, you’ll see the “command not found” error.

1. Install Python: If Python is not installed, you can download and install it from the official Python website. Go to python.org/downloads and download the latest version for macOS. Follow the installation instructions provided on the website.

2. Verify Python Installation: After installing Python, verify the installation by running the python –version command again. If it displays the installed Python version, Python is now installed correctly.

3. Check PATH Variable: If Python is installed but you’re still encountering the “command not found” error, it’s possible that Python’s directory is not included in your system’s PATH variable. To check your PATH variable, type the following command in Terminal:

echo $PATH

This will display a list of directories separated by colons (:). If Python’s directory (usually something like /usr/local/bin) is not included in the PATH, you’ll need to add it.

1. Add Python to PATH: You can add Python’s directory to your PATH by editing your shell configuration file (.bash_profile, .bashrc, .zshrc, etc.). Open the appropriate file in a text editor (you can use nano, vim, or any text editor you prefer) and add the following line:

export PATH="/usr/local/bin:$PATH"

Replace /usr/local/bin with the actual path to your Python installation directory if it’s different. Save the file and then restart your Terminal or run source ~/.bash_profile (or source ~/.bashrc, etc.) to apply the changes.

After following these steps, try running python –version again. If Python is installed correctly and included in your PATH, you should see the installed Python version without any errors.

How to fix “ModuleNotFoundError: No module named ‘requests’”

If you encounter the error “ModuleNotFoundError: No module named ‘requests’” it means that the requests module is not installed in your Python environment. Here’s how you can fix it:

1. Install requests Module: You can install the requests module using pip, which is the package installer for Python. Open your terminal and run the following command:

pip install requests

This command will download and install the requests module and its dependencies. Depending on your system configuration, you might need to use pip3 instead of pip if you’re using Python 3.

1. Verify Installation: After installing the requests module, you can verify that it’s installed correctly by running a Python interpreter and trying to import it. Open your terminal and run:

python

This will start the Python interpreter. Then, try to import requests: If you don’t see any error messages, the requests module is installed successfully.

By following these steps, you should be able to resolve the “ModuleNotFoundError: No module named ‘requests’” error and use the requests module in your Python scripts.

1. Check Python Environment: Make sure you’re installing the requests module in the correct Python environment. If you’re using virtual environments (created with virtualenv, venv, or conda, for example), ensure that you’ve activated the environment before running pip install requests.