
Vibe Coding With Claude Desktop and MCP Part 2 - Switching to Scrapy
In my last post, I had a TypeScript-based scraper. In this one I switch to Scrapy.
I use nvm. So, maybe that why I had so many issues. I started with one tutorial that told me all I had to do was run these commands:
npx -y @smithery/cli install @wonderwhy-er/desktop-commander --client claude
npx -y @smithery/cli@latest install @smithery-ai/server-sequential-thinking --client claude
And that didn’t work for me. I got a Could not attach to MCP server filesystem
error.
Then another that told me all I had to do was create a wrapper for nvm and use that. Same error.
This is what finally worked for me. Run these commands to install the MCP servers globally:
npm install -g @modelcontextprotocol/server-sequential-thinking
npm install -g @wonderwhy-er/desktop-commander
Then open Claude’s settings:
Then open the Config file manually by clicking Edit Config in the Developer section:
This will open the folder with claude_desktop_config.json
selected. Open the file and hard code the paths of your node installation and the index file of the dist folder (at least for these two MCP servers) .
{
"mcpServers": {
"server-sequential-thinking": {
"command": "/Users/YOUR_USERNAME/.nvm/versions/node/v20.16.0/bin/node",
"args": [
"/Users/YOUR_USERNAME/.nvm/versions/node/v20.16.0/lib/node_modules/@modelcontextprotocol/server-sequential-thinking/dist/index.js"
]
},
"desktop-commander": {
"command": "/Users/YOUR_USERNAME/.nvm/versions/node/v20.16.0/bin/node",
"args": [
"/Users/YOUR_USERNAME/.nvm/versions/node/v20.16.0/lib/node_modules/@wonderwhy-er/desktop-commander/dist/index.js"
]
}
}
}
You can the location of the node installation you are using by running this command:
which node
Which will return something like this:
/Users/YOUR_USERNAME/.nvm/versions/node/v20.16.0/bin/node
Just replace YOUR_USERNAME
in the config file with your actual username and the version number with the version number of your node installation. And it worked. Then I needed something to build.
I have had this idea for a new project for a while and just kept putting it on the back burner because I knew it would take some time. It seemed like a good choice for this experiment:
There is actually more I plan on doing after that, but that should get and keep things going.
Tips:
The first night, I messed with the setup and then only had about a half hour left, so I worked on the Gatsby part of the app. I already had a basic installation but hadn’t made any changes to it. And if you’ve ever done that, you know how basic it is. This my first prompt and I really didn’t expect much:
In /path/my/project/is/in, I am building a Gatsby site. I want a nice looking theme and I want the standard blog+pages type of structure.
Each time it accesses a new folder, it will ask for permission. Sometimes it seems like it asks way too many times but it eventually stops. I actually forgot to take this screenshot, so I created another project to update the theme on my Jekyll blog which is over a decade old. In that case, it actually told me to create another branch in git to work on those changes.
Once you allow it to access a folder, its off and running:
The I just told it to Continue
a few times. When there was an error, I told it about it and it fixed it. This is where I ran into the node version mismatch, but once I told it my version, it fixed things. When it was done, it told me how to run it:
So I wasn’t sure what to expect but here’s what I got when I ran it:
A little bit bright and primary colored for me, but a good start. So I told it:
I want to make the colors in the theme more mellow. Right now they are bold and primary colors.
Not much of a prompt, but it gave me this, which is better. I will tweak it later, but I wanted to get onto other things:
I also told it:
What if in the future, I wanted to scrape some data from websites and use that data to generate pages. These will be writing markets, so I would probably want to create a different type of page for them.
After some back and forth on what format I wanted to store each market as, I choose a JSON file per market and got this and the filters and search work well. It just added that without me asking for it:
The second night, I also had around an hour. This night I wanted to create a scraper to provide data for the site. This was my first prompt:
I am building a gatsby project here: /my/project/directory for a blog and to list writing markets. You can see the structure in /my/project/directory/src/data/markets/sample-market.json. That may have to change as I go, as well as its templates. But for now I want to get started on a webscraper project to generate the json files for the gatsby site in /my/project/directory/data. I am not sure what to use for that just yet. I want to be able to eventually run it on cron.
I gave me a project summary and asked me if I wanted to continue.
I will probably see if it will refactor this with Scrapy later and see what happens, but I told it that was find and all I did after that was tell it to continue
4 or 5 times and here is the final result:
And it did kind of work. I didn’t guide it very much and it was a good start. It scraped some markets and put it in the folder in JSON format, but only about 8. But like I said, I wanted to see what it would do without much guidance.
I am actually going to work on this a little more tonight. Some things I’m thinking of include:
It still has a long way to go but this is only part one and I’ll write about it as I go.
Part 2 is here: Vibe Coding With Claude Desktop and MCP Part 2 - Switching to Scrapy