Public forms can't be deleted since they are your pipe's start form. However, you can disable the public form so that external people will no longer be able to access it through the public URL.
You can disable the form in two ways:
1. Click on the Share button on the pipe header. Then, click on the Public button and the green key will turn gray - which means that the form has been disabled.
2. Another way to disable the form is to click on the Form button, in the header of the pipe. You will see the form's editing page, just click on the Public button to disable it.