ImportError: cannot import name 'Client' from scrapegraph_py when using scrapegraph-py 2.1.0

[adithyanajay]

Describe the bug

The cognee-community-tasks-scrapegraph package is not compatible with the latest scrapegraph-py (v2.1.0).

When importing scrape_urls from cognee_community_tasks_scrapegraph, the import fails with:

ImportError: cannot import name 'Client' from 'scrapegraph_py'

The package appears to expect the older scrapegraph-py API (Client), while the latest SDK (v2.1.0) no longer exposes that class.

---

To Reproduce

1. Create a new Python virtual environment.

2. Install:

   pip install cognee-community-tasks-scrapegraph scrapegraph-py

3. Create a simple script:

   import asyncio
   import os
   from cognee_community_tasks_scrapegraph import scrape_urls
   
   os.environ["SGAI_API_KEY"] = "your-api-key"
   
   async def main():
       results = await scrape_urls(
           urls=[
               "https://cognee.ai",
               "https://docs.cognee.ai",
           ],
           prompt="Extract the product description, key features, and target use cases",
       )
   
       print(results)
   
   asyncio.run(main())

4. Run the script.

---

Actual behavior

The script fails during import with:

ImportError: cannot import name 'Client' from 'scrapegraph_py'

Traceback points to:

from scrapegraph_py import Client

inside cognee_community_tasks_scrapegraph/scrapegraph_task.py.

---

Expected behavior

The package should either:

• support the latest scrapegraph-py SDK (v2.x), or
• specify a compatible version constraint (for example, pinning scrapegraph-py<2 if only the older API is supported).

---

Screenshots

---

Desktop

• OS: Ubuntu 24.04 LTS
• Python: 3.12
• cognee-community-tasks-scrapegraph: 0.1.1
• scrapegraph-py: 2.1.0

---

Smartphone

Not applicable.

---

Additional context

Installed package versions:

cognee-community-tasks-scrapegraph==0.1.1
scrapegraph-py==2.1.0

It appears that scrapegraph_task.py still imports:

from scrapegraph_py import Client

However, Client is no longer available in scrapegraph-py v2.x, resulting in an immediate ImportError before any user code can execute.

[dosubot]

Hi @adithyanajay! I'm Dosu and I'm helping the ScrapeGraphAI, Inc team.

The Client class still exists in scrapegraph-py v2.x — it was not removed, but its methods were significantly refactored in PR #1058. Specifically, in v2.x the API changed as follows:

v1 Method	v2 Method
smartscraper(website_url=, user_prompt=)	extract(url=, prompt=, output_schema=)
searchscraper(user_prompt=)	search(query=)
markdownify(website_url=)	scrape(url=)
get_credits()	credits()

So from scrapegraph_py import Client should still work with scrapegraph-py==2.1.0. If you're getting an ImportError, it's possible there's a packaging or installation conflict in your environment.

That said, the cognee-community-tasks-scrapegraph package is not part of this repository — it's a third-party community package. The incompatibility you're experiencing is in that external package's code, not in scrapegraph-py itself. The fix would need to come from the maintainers of cognee-community-tasks-scrapegraph.

As a workaround, you could try pinning to an older version:

pip install "scrapegraph-py<2"

Or, if you don't need the cognee-community-tasks-scrapegraph wrapper, you can use scrapegraph-py v2.x directly [1]:

from scrapegraph_py import Client

client = Client(api_key="your-api-key")
result = client.extract(url="https://cognee.ai", prompt="Extract the product description, key features, and target use cases")

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.