Skip to content

Setting up CGI

Web hosting discussion, programming, and shared and dedicated servers.
6 posts Page 1 of 1
Post a reply

Setting up CGI

by ejwells » Fri Aug 15, 2014 6:58 pm
I am trying to set up a simple Python CGI script, but nothing I've tried works. I searched all the Forums and Wiki pages but couldn't find anything that helped. I have both Mini Hosting and Personal Hosting but neither of these setups worked.

The Python script is "HelloWorld.py" as follows:

Code: Select all

#!/usr/bin/python

print "Content-Type: text/html"
print ""
print "<html>"
print "<body>"
print "<h1>Hello World!</h1>"
print "</body>"
print "</html>"
(Permissions are rwx-r-x-r-x)

Under my home directory I have two links that seem to have been created automatically:

Code: Select all

lrwxrwxrwx    1 root     root           36 Aug 13 21:12 /home/e/ejwells/public_cgi -> /usr/local/lib/httpd/cgi-bin/ejwells
lrwxrwxrwx    1 ejwells  user           23 Jul  6  2007 /home/e/ejwells/public_html -> /home/WWW_pages/ejwells
(There are others, which I assume are irrelevant for hosting CGI)

I also created an ".htaccess" file as suggested in the Wiki page and a previous posting on this subject:

Code: Select all

AddHandler cgi-script .cgi .pl .py
(With AND without a blank line on the end. Permissions are rw--r---r--, but also tried with execute added)
I tried with this file in both /home/WWW_pages/ejwells and with a copy in every directory under my home.

Under the Mini Hosting setup WITHOUT .htaccess:
Script Location: /home/WWW_pages/ejwells/HelloWorld.py
URL: http://ejwells.users.sonic.net/HelloWorld.py
Result: Treated as "plain text" (as expected)

URL: http://ejwells.users.sonic.net/cgi/HelloWorld.py
Result: "CGIwrap Error: User not found in passwd file."

URL: http://ejwells.users.sonic.net/cgi-bin/HelloWorld.py
Result: "Not Found The requested URL /cgi-bin/HelloWorld.py was not found on this server."

URL: http://ejwells.users.sonic.net/cgi/ejwe ... loWorld.py
Result: "CGIwrap Error: Script File Not Found!"

URL: http://ejwells.users.sonic.net/cgi-bin/ ... loWorld.py
Result: "Not Found The requested URL /cgi-bin/HelloWorld.py was not found on this server."
Under the Mini Hosting setup WITH ~/.htaccess:
Script Location: /home/WWW_pages/ejwells/HelloWorld.py
URL: http://ejwells.users.sonic.net/HelloWorld.py
Result: "Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request."

URL: http://ejwells.users.sonic.net/cgi/HelloWorld.py
Result: "CGIwrap Error: User not found in passwd file."

URL: http://ejwells.users.sonic.net/cgi-bin/HelloWorld.py
Result: "Not Found The requested URL /cgi-bin/HelloWorld.py was not found on this server."

URL: http://ejwells.users.sonic.net/cgi/ejwe ... loWorld.py
Result: "CGIwrap Error: Script File Not Found!"

URL: http://ejwells.users.sonic.net/cgi-bin/ ... loWorld.py
Result: "Not Found The requested URL /cgi-bin/HelloWorld.py was not found on this server."

Script Location: /usr/local/lib/httpd/cgi-bin/ejwells
URL: http://ejwells.users.sonic.net/HelloWorld.py
Result: "Not Found The requested URL /HelloWorld.py was not found on this server." (As expected)

URL: http://ejwells.users.sonic.net/cgi/HelloWorld.py
Result: "CGIwrap Error: User not found in passwd file."

URL: http://ejwells.users.sonic.net/cgi-bin/HelloWorld.py
Result: "Not Found The requested URL /cgi-bin/HelloWorld.py was not found on this server." (Unexpected)

URL: http://ejwells.users.sonic.net/cgi/ejwe ... loWorld.py
Result: "CGIwrap Error: System Error: execv() failed Error: No such file or directory (2)"

URL: http://ejwells.users.sonic.net/cgi-bin/ ... loWorld.py
Result: "Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request."
In summary: using "cgi-bin" in the URL seems to result in "Not Found" errors (despite what is stated in the Wiki and Forum postings). Whenever the script can be accessed via URL it seems to give some sort of "permission/execute" error.

I tested some other combinations, and using my Personal Hosting site, but still got the above errors. It's quite possible there's something I'm doing wrong, but I cannot figure out what!

When I can get the Mini Hosting site to work I will also need to get CGI working on my Personal Hosting site also. I don't mind (too much) if the CGI scripts have to use the same directory, but it would be better if I could keep them in separate directories.

It would REALLY HELP if there was a simple setup guide for CGI that clearly stated what files were needed, what their contents should be, and where they should be located.

Any and all help is appreciated as I've wasted so much time on this!

Best Regards,
Eric.
by joemuller » Sat Aug 16, 2014 5:13 pm
Ah, the dark arts of CGI...

First, we do have a CGI FAQ, though it needs some updating - shoot me a PM if you see anything out of place - I'll be glad to make corrections.

But, let's see if I can help you out. The easiest way to get CGI working without having to move files all over the place is to edit/update the .htaccess file in your public_html folder as you did above:

Code: Select all

AddHandler cgi-script .cgi .pl .py
The next step is to make sure the executable ('x') permission is set for the file. (Set via properties in your FTP client or 'chmod +x FILENAME' at the shell.)

On any of the hosting packages, you'll probably want to have your CGI scripts live in the public_cgi folder. You can then call or access these scripts by adding /cgi/ or /cgi-bin/ before their name. (public_cgi is mapped to /cgi and /cgi-bin)

Go ahead and give these a shot and let me know how it goes.

-- Joe
I'm a proud employee of Sonic.net! :-)
by ejwells » Sun Aug 17, 2014 2:33 pm
Thanks for the response, but did you actually READ my post?

I have done all you suggested (and more) and even posted the results of some of my testing:

Code: Select all

$ ls -alL ~/public_cgi ~/public_html
/home/e/ejwells/public_cgi:
total 292
drwx------    2 ejwells  user         4096 Aug 15 18:15 .
drwxr-xr-x  3346 root     root       286720 Aug 14 19:09 ..
-rwxr-xr-x    1 ejwells  user          159 Aug 15 15:55 HelloWorld.py

/home/e/ejwells/public_html:
total 764
drwx------    4 ejwells  user         4096 Aug 15 18:15 .
drwxr-xr-x  9619 root     root       757760 Aug 17 06:27 ..
-rwxr-xr-x    1 ejwells  user           36 Aug 15 16:08 .htaccess
-rw-r--r--    1 ejwells  user          515 Aug 13 19:20 index.html
$
(not all shown)

Code: Select all

$ cat ~/public_html/.htaccess
AddHandler cgi-script .cgi .pl .py
$
Attempting to access the script: http://ejwells.users.sonic.net/cgi/HelloWorld.py gives:
CGIwrap Error: User not found in passwd file.
Using http://ejwells.users.sonic.net/cgi-bin/HelloWorld.py gives:
Not Found
The requested URL /cgi-bin/HelloWorld.py was not found on this server.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
Apache/1.3.41 Server at ejwells.users.sonic.net Port 80
Also, as originally stated, The FAQ's are out of date and incomplete. In my opinion they should be completely rewritten. The Forums only seem to contain (some) fixes for specific problems.

What is needed is a setup guide, not just FAQ's. There is little point in providing these sort of tools (CGI, MySQL, Wordpress,etc) if you don't tell people how to use them and won't support them by telephone.

Please read my original post and let me know if there's anything you need to know that I haven't already stated.

Best Regards,
Eric.
by joemuller » Mon Aug 18, 2014 4:29 pm
I've found a solution for you, or rather, two fixes.

1.) The /cgi and /cgi-bin path only works for virtual hosting - the sonic.net/~USER of USER.users.sonic.net URLs will give the 404 and CGI-wrapper errors you were seeing before. If you move your CGI script to a location inside your public_html directory, it will work. (Assuming you added the 'AddHandler cgi-script .cgi .pl .py' line to your .htaccess.)

2.) The reason your script was failing was because it did not end correctly. To be more precise, the last line did not contain a newline, so CGI just threw up its hands and gave up. The reason why your file didn't have the necessary newline (usually invisible) is because it was created with DOS formatting. The way this usually happens is by uploading script file over FTP, but using the 'BINARY' mode instead of 'TEXT'.

As for your existing file, you can fix it in one of two ways - either add an extra blank line on the end, or you can use the dos2unix tool on the Sonic.net shell server.
I'm a proud employee of Sonic.net! :-)
by ejwells » Fri Aug 22, 2014 4:57 pm
Thanks for the more considered response!

It did not quite fix the problem but it gave me a pointer to the problem(s)...
  1. Not only does the "/cgi" & "/cgi-bin" mapping only work for Virtual Hosting, but it seems to prevent you using this path for other types of hosting. I.E. you cannot have a "cgi" directory under "~/public_html" or link to "~/public_cgi" unless you have a Virtual Hosting setup.

    (I am not completely sure about this because I do not have Virtual Hosting, but everything I tried with "/cgi" gave either a "404 - Not Found" or a "User not found in passwd file" error. I have Mini Hosting and personal Hosting).

    Conclusion: If you're not using Virtual Hosting scripts MUST be under "~/public_html" and you must have a .htaccess file (see below) in the same directory.
  2. There was a problem with transferring files from Windows via FTP.

    I had added a blank line to the end of the Python scripts, but this was not enough because the other lines in the file still had the wrong CR/LF combination. (I only discovered this after recreating the scripts at the shell prompt).

    What made matters worse is I had tested all the scripts by running them with Python at the shell prompt. However this accepts the Windows CR/LF and all the scripts ran OK! They ONLY failed when run through the Apache cgi-script handler!

    Conclusion: If you get a "500 - Internal Server Error" check your scripts for errors. Always run "dos2unix" if you are transferring files from Windows/DOS
Unfortunately the limitations in [1] mean that I'm not going to spend any more time on this. It is just the start of a project to implement a simple web service and the Sonic hosting is just making life too difficult. I will probably get a simple Virtual Machine from AWS where I can control the setup directly.

Thanks for the help.

Best Regards,
Eric.
by aaazen » Sun Sep 14, 2014 11:37 pm
ejwells wrote:
  1. Not only does the "/cgi" & "/cgi-bin" mapping only work for Virtual Hosting, but it seems to prevent you using this path for other types of hosting. I.E. you cannot have a "cgi" directory under "~/public_html" or link to "~/public_cgi" unless you have a Virtual Hosting setup.

    (I am not completely sure about this because I do not have Virtual Hosting, but everything I tried with "/cgi" gave either a "404 - Not Found" or a "User not found in passwd file" error. I have Mini Hosting and personal Hosting).

    Conclusion: If you're not using Virtual Hosting scripts MUST be under "~/public_html" and you must have a .htaccess file (see below) in the same directory.
The conclusion is not true. One can create a subdirectory other than cgi-bin or cgi such as bin or cgibin and place python scripts there. Under the subdirectory one must also place an .htaccess file.

I have placed the script and an .htaccess file under my ~/public_html/cgibin/ directory:
http://aaazen.users.sonic.net/cgibin/temp.py
ejwells wrote:
  • There was a problem with transferring files from Windows via FTP.

    I had added a blank line to the end of the Python scripts, but this was not enough because the other lines in the file still had the wrong CR/LF combination. (I only discovered this after recreating the scripts at the shell prompt).

    What made matters worse is I had tested all the scripts by running them with Python at the shell prompt. However this accepts the Windows CR/LF and all the scripts ran OK! They ONLY failed when run through the Apache cgi-script handler!

    Conclusion: If you get a "500 - Internal Server Error" check your scripts for errors. Always run "dos2unix" if you are transferring files from Windows/DOS
The 500 Internal Server Error can occur if there are tabs or too many leading spaces in a python script as well.

It sounds like the testing was incomplete.

Python will accept a CR/LF at the end of each line, but Linux/Unix will not.

I tested the temp.py script from a Linux command line like this:

$cd ~/public_html/cgibin/
$./temp.py

When temp.py contained CR characters then the error message was this:

./temp.py: /usr/bin/python^M: bad interpreter: No such file or directory

The ^M is a CR and since on Unix a CR is just another character,
Bash is looking for a script language called /usr/bin/python^M

--------------------------------------------------------------------------------------
2014-09-15: 08:33 PDT
For completeness here are all the files involved:

Code: Select all

aaazen@bolt:~(0)$ ls -altr public_html/cgibin/
total 12
drwx------    3 aaazen   user         4096 Sep 13 13:50 ..
-rw-r--r--    1 aaazen   user           35 Sep 14 22:21 .htaccess
-rwxr-xr-x    1 aaazen   user          152 Sep 14 23:08 temp.py
drwxr-xr-x    2 aaazen   user         4096 Sep 15 08:23 .
aaazen@bolt:~(0)$ cat public_html/cgibin/.htaccess
AddHandler cgi-script .cgi .pl .py
aaazen@bolt:~(0)$ cat public_html/cgibin/temp.py
#!/usr/bin/python

print "Content-Type: text/html"
print ""
print "<html>"
print "<body>"
print "<h1>Hello World!</h1>"
print "</body>"
print "</html>"
6 posts Page 1 of 1

Return to “Hosting”