<?xml version="1.0"?>

<st-source>
<!-- 
Name: WebSupport
Notice: Copyright © 2008 Cincom Systems, Inc.  All Rights Reserved.
Comment: This package provides HttpClient and HttpRequest extensions simplifying submission of HTML form data through HTTP POST and GET methods. Form data can be submitted in a "url encoded" format in a simple, single-part HTTP request (content-type: application/x-www-form-urlencoded). Alternatively each data entry can be submitted as an individual part in a multipart HTTP request (content-type: multipart/form-data). Multipart messages are used when form data contains entries with relatively large values, for example when a form has external files attached to it for upload to the server. More information about HTML forms can be found at http://www.w3.org/TR/html401/interact/forms.html#h-17.13.

	Submitting form data using POST method.

The default behavior is to submit forms as simple requests. Form entries can be added individually using #addFormKey:value: message, or set at once using #formData: message which takes a collection of Associations. Note that #formData: replaces any previous form content.

	stream := String new writeStream.
 	(HttpRequest post: 'http://localhost/xx/ValueOfFoo')
		addFormKey: 'foo' value: 'bar';
		addFormKey: 'file'  value: 'myFile';
		writeOn: stream.
	stream contents

An alternative way to post a form is through HttpClient, in this case the request gets automatically executed and the result is the response from the server.

	HttpClient new
		post: 'http://localhost/xx/ValueOfFoo' 
		formData: (
			Array
				with: 'foo' -> 'bar';
				with:'file' -> 'myFile').

To force the form to submit as a multipart message, send #beMultipart to the request at any point. Any previously added entries will be automatically converted to message parts. Note however that conversion of multipart messages back to simple messages is not supported, as it is not always possible without potentially losing information.

	stream := String new writeStream.
 	(HttpRequest post: 'http://localhost/xx/ValueOfFoo')
		addFormKey: 'foo' value: 'bar';
		beMultipart;
		addFormKey: 'file'  value: 'myFile';
		writeOn: stream.
	stream contents

File entries can be added using message #addFormKey:filename:source:. Adding a file entry automatically forces the message to become multipart to be able to capture both the entry key and the filename.

	stream := String new writeStream.
	(HttpRequest post: 'http://localhost/xx/ValueOfFoo')
		addFormKey: 'foo' value: 'bar';
		addFormKey: 'text'  filename: 'text.txt' source: 'some text' readStream;
		writeOn: stream.
	stream contents

Adding a file entry attempts to guess the appropriate Content-Type for that part from the filename extension. If it doesn't succeed the content type is set to default, i.e application/octet-stream. File names with non ASCII character will be automatically encoded using UTF8 encoding. UTF8 will also be used for the file contents if the source is a character stream (as opposed to byte stream).

Adding an entry to a multipart message returns the newly created part. That allows to modify any of the default settings or to add new ones. Here's an example changing the filename and file contents encoding to ISO8859-2:

	stream := String new writeStream.
	request := HttpRequest post: 'http://localhost/xx/ValueOfFoo'.
	part := request addFormKey: 'czech'
				filename: 'kúň.txt'
				source: 'Příliš žluťoučký kúň úpěl ďábelské ódy.' withCRs readStream.
	part headerCharset: #'iso-8859-2';
		charset: #'iso-8859-2'.
	request writeOn: stream.
	stream contents

There's also an API to parse messages containing forms in any of the supported forms. Just send #formData to the HTTP message. The result is a collection of associations, the same form as the input to the #formData: message.

 	(HttpRequest post: 'http://localhost/xx/ValueOfFoo')
		addFormKey: 'foo' value: 'bar';
		addFormKey: 'file'  value: 'myFile';
		formData

File entry values will be entire message parts so that all the associated information can be accessed.

	request := (HttpRequest post: 'http://localhost/xx/ValueOfFoo')
		addFormKey: 'foo' value: 'bar';
		addFormKey: 'text'  filename: 'text.txt' source: 'some text' readStream;
		yourself.
	part := request formData last value.
	part contents


	Submitting form data using GET method.

HttpClient can submit a query using GET method as well. The request gets automatically executed and the result is the response from the server.
	response := HttpClient new
		get: 'http://www.google.ca/search'
		formData: (Array with: 'q' -> 'Cincom').


Form entries can be added individually using #addFormKey:value: message, or set at once using #formData: message which takes a collection of Associations. Note that #formData: replaces any previous form content.

	stream := String new writeStream.
	request := HttpRequest get: 'http://www.google.ca/search'.
	request addFormKey: 'q' value: 'WebSupport';
		addFormKey: 'ie' value: 'iso8859-1';
		addFormKey: 'oe' value: 'utf8';
		addFormKey: 'client' value: 'VW';
		yourself.
	request writeOn: stream.
DbIdentifier: bear73
DbTrace: 119608
DevelopmentPrerequisites: #(#(#any 'HTTP' ''))
PackageName: WebSupport
Parcel: #('WebSupport')
PrerequisiteParcels: #(#('HTTP' ''))
PrintStringCache: (751 11,mkobetic)
Version: 7.6
Date: 12:34:15 pm February 1, 2008
 -->
<time-stamp>From VisualWorks®, 7.6 of February 1, 2008 on February 1, 2008 at 12:34:15 pm</time-stamp>


<do-it>(Dialog confirm: 'You are filing-in a Parcel source file!\\While this is possible it will not have\the same effect as loading the parcel.\None of the Parcel''s prerequisites will\be loaded and none of its load actions\will be performed.\\Are you sure you want to file-in?' withCRs) ifFalse: [self error: 'Parcel file-in abandoned.  Choose terminate or close.']</do-it>

<class>
<name>WebClient</name>
<environment>Net</environment>
<super>Net.HttpClient</super>
<private>false</private>
<indexed-type>none</indexed-type>
<inst-vars></inst-vars>
<class-inst-vars></class-inst-vars>
<imports></imports>
<category>WebSupport</category>
<attributes>
<package>WebSupport</package>
</attributes>
</class>

<comment>
<class-id>Net.WebClient</class-id>
<body>Currently all the functionality this used to provide (and more) is now available as extensions on its superclass (HttpClient) in this package.

</body>
</comment>

<methods>
<class-id>Net.HttpClient</class-id> <category>commands</category>

<body package="WebSupport" selector="get:formData:">get: anURL formData: aCollectionOfAssociations
"Compose and send a GET query"
	| data |
	data := (aCollectionOfAssociations isKindOf: Dictionary)
			ifTrue: [aCollectionOfAssociations associations]
			ifFalse: [aCollectionOfAssociations].
	(request := self requestClass get: anURL)
		formData: data.
	^self executeAndGetResponse.</body>

<body package="WebSupport" selector="post:formData:">post: anURL formData: aCollectionOfAssociations
	"Send form data sets as prescribed by http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4"
	| data |
	data := (aCollectionOfAssociations isKindOf: Dictionary)
			ifTrue: [aCollectionOfAssociations associations]
			ifFalse: [aCollectionOfAssociations].
	(request := self requestClass method: 'POST' url: anURL)
		formData: data.
	^self executeAndGetResponse.</body>
</methods>

<methods>
<class-id>Net.HttpRequest</class-id> <category>post forms</category>

<body package="WebSupport" selector="addFormKey:filename:source:">addFormKey: aString filename: fileName source: aStream
"This method can be used to attach files for upload. We don't follow the multipart/mixed recommendation below, because none of the widely used web browsers seem to follow it either and there's consequently a danger that some servers might have troubles processing it as well.

From http://www.w3.org/TR/html401/interact/forms.html#h-17.13:
If the contents of a file are submitted with a form, the file input should be identified by the appropriate content type (e.g., application/octet-stream). If multiple files are to be returned as the result of a single form entry, they should be returned as multipart/mixed embedded within the multipart/form-data.

The user agent should attempt to supply a file name for each submitted file. The file name may be specified with the filename parameter of the 'Content-Disposition: form-data' header, or, in the case of multiple files, in a 'Content-Disposition: file' header of the subpart. If the file name of the client's operating system is not in US-ASCII, the file name might be approximated or encoded using the method of [RFC2045]. This is convenient for those cases where, for example, the uploaded files might contain references to each other (e.g., a TeX file and its .sty auxiliary style description)."

	| part tokens  |
	self method = 'POST' 
		ifFalse: [^self error: #OnlyPOSTUploadFiles &lt;&lt; #net &gt;&gt; 'Only POST method is used to upload files!'].
	self ensureMultipartForm.
	part := Net.MimeEntity new.
	"extension guessing is sensitive to non ascii characters in the filename (due to the use of asFilename), so let's feed the guesser just the extension for more resilience."
	tokens := fileName tokensBasedOn: $..
	part contentType: (tokens size &gt; 1
			ifTrue: [MimeTypeDescriptor guessFromExtension: tokens last]
			ifFalse: [MimeTypeDescriptor defaultUnsupportedContentType]).
	part headerCharset: #'utf-8'.
	(part contentDisposition: 'form-data') 
		parameterAt: 'name' put: aString;
		fileName: fileName.   
	aStream isCharacterStream
		ifTrue: [ 	part charset:  #'utf-8'.
				part body source: aStream]
		ifFalse: [part body byteSource: aStream.].
	self addPart: part.
	self contentType: 'multipart/form-data'.
	^part</body>

<body package="WebSupport" selector="addFormKey:value:">addFormKey: keyString value: valueString
"Use the method to add form data to url encoded and multipart forms.
By default the form is created as urlencoded. Use #beMultipart to switch to  multipart/form-data " 

	self method = 'POST'
		ifTrue: [^self isMultipart
					ifTrue: [ self addMultipartFormKey: keyString value: valueString]
					ifFalse: [self addSimpleFormKey: keyString value: valueString]].

	self method = 'GET'
		ifTrue: [^self addURLFormKey: keyString value: valueString ].

	self wrongHttpMethodForFormSubmission</body>
</methods>

<methods>
<class-id>Net.HttpRequest</class-id> <category>private post forms</category>

<body package="WebSupport" selector="addMultipartFormKey:value:">addMultipartFormKey: keyString value: valueString
	"Creates multipart message "
	| part |
	(part := Net.MimeEntity new)
		addField: ContentDispositionField formData.
	part contentDisposition parameterAt: 'name' put: keyString.
	part body byteSource:  (valueString asByteArrayEncoding: #utf8) readStream.
	self addPart: part.
	self contentType: 'multipart/form-data'.
	^part</body>

<body package="WebSupport" selector="addSimpleFormKey:value:">addSimpleFormKey: keyString value: valueString
	"Creates  urlencoded simple body"
	| stream |
	self contentType: 'application/x-www-form-urlencoded'.
	self body source
		ifNil: [ self body source: (ReadWriteStream on: String new) ]
		ifNotNil: [ self body source nextPut: $&amp; ].
	stream := self body source.
	URLEncoder new
		encode: keyString
		onto: stream.
				
	stream nextPut: $=.
	URLEncoder new
		encode: valueString
		onto: stream</body>

<body package="WebSupport" selector="addURLFormKey:value:">addURLFormKey: keyString value: valueString
	"Creates  urlencoded simple body"
	| stream |
	stream := String new writeStream.
	self url query ifNotNil: [ :query |
		stream nextPutAll: query;
		nextPut: $&amp; ].
	URLEncoder new
		encode: keyString copy
		onto: stream.
	stream nextPut: $=.
	URLEncoder new
		encode: valueString copy
		onto: stream.
	self url query: stream contents</body>
</methods>

<methods>
<class-id>Net.HttpRequest</class-id> <category>post forms</category>

<body package="WebSupport" selector="beMultipart">beMultipart
"Switch to multipart content type "

	self bot makeMultipart: self</body>
</methods>

<methods>
<class-id>Net.HttpRequest</class-id> <category>private post forms</category>

<body package="WebSupport" selector="decodedFormContentsOf:">decodedFormContentsOf: part
"Apparently browsers (at least Firefox) don't provide any hint in the response about the encoding they uses for the part body.
Our HTTP layer therefore assumes the spec prescribed ISO 8859-1.
We'll force utf8 here for decoding." 

	| contents charset |
	charset := part charsetIfNone: [ #utf8 ].
	contents := part body valueAsStream.
	^contents isExternalStream
		ifTrue: [ self decodedFormContentsOfExternalStream: contents charset: charset ]
		ifFalse: [ self decodedFormContentsOfStream: contents charset: charset ]</body>

<body package="WebSupport" selector="decodedFormContentsOfExternalStream:charset:">decodedFormContentsOfExternalStream: stream charset: charset

	| wasBinary |
	
 	wasBinary := stream isBinary.
	stream binary.
	^[ self decodedFormContentsOStream: stream charset: charset
	] ensure: [ wasBinary ifFalse: [ stream text ] ]</body>

<body package="WebSupport" selector="decodedFormContentsOfStream:charset:">decodedFormContentsOfStream: stream charset: charset

	^(EncodedStream
			on: stream
			encodedBy: (StreamEncoder new: charset )
	) contents</body>

<body package="WebSupport" selector="ensureMultipartForm">ensureMultipartForm

	| formData |
	self contentType = 'application/x-www-form-urlencoded' 
		ifTrue:[ formData := self formData.
				self beMultipart.
				self removePart: self parts first.
				self formData: formData.].</body>
</methods>

<methods>
<class-id>Net.HttpRequest</class-id> <category>post forms</category>

<body package="WebSupport" selector="formData">formData

	^self method = 'POST'
		ifTrue: [ self isMultipart
				ifTrue: [ self multipartFormData ]
				ifFalse: [ self simpleFormData ] ]
		ifFalse: [ self urlEncodedDataFrom: self url query ]</body>

<body package="WebSupport" selector="formData:">formData: aCollectionOfAssociations
"Use the method to create url encoded and multipart forms.
By default the form is created as urlencoded. Use #beMultipart to switch to  multipart/form-data " 

	(#('GET' 'POST') includes: self method) ifFalse: [ ^self wrongHttpMethodForFormSubmission ].
	self method = 'POST'
		ifTrue: [ 	self value: nil ]
		ifFalse: [self url query: nil].

	aCollectionOfAssociations
		do: [ :assoc | self addFormKey: assoc key value: assoc value ]</body>
</methods>

<methods>
<class-id>Net.HttpRequest</class-id> <category>private post forms</category>

<body package="WebSupport" selector="multipartFormData">multipartFormData

	| collection name |
	collection := OrderedCollection new.
	self parts do: 
		[ :part |
		name := part contentDisposition parameterAt: 'name' ifAbsent: [String new].
		collection add: ( name-&gt; (part formFilename
									ifNil: [ self decodedFormContentsOf: part ]
									ifNotNil: [part])) ]	.

	^collection</body>

<body package="WebSupport" selector="simpleFormData">simpleFormData

	^self urlEncodedDataFrom: self contents</body>

<body package="WebSupport" selector="urlEncodedDataFrom:">urlEncodedDataFrom: aString

	^aString
		ifNil: [ OrderedCollection new ]
		ifNotNil: [ :contents || rs fields |
			fields := OrderedCollection new.
			rs := contents readStream.
			[ rs atEnd ] whileFalse: [ | field key val mid |
				field := rs upTo: $&amp;.
				(mid := field indexOf: $=) isZero
					ifTrue: [ key := field. val := '' ]
					ifFalse: [ key := field copyFrom: 1 to: mid - 1.
						val := field copyFrom: mid+1 to: field size ].
				key := URLEncoder decode: key.
				val := URLEncoder decode: val.
				fields add: key-&gt;val.].
			fields ]</body>

<body package="WebSupport" selector="wrongHttpMethodForFormSubmission">wrongHttpMethodForFormSubmission

	self error: #WrongHTTPMehthodForForm &lt;&lt; #net &gt;&gt; 'Only POST and GET requests can be used for form/query submission!'</body>
</methods>

<methods>
<class-id>Net.MimeEntity</class-id> <category>html forms</category>

<body package="WebSupport" selector="formFilename">formFilename

	^(self fieldAt: 'content-disposition' ifAbsent: [^nil]) fileName</body>
</methods>

<do-it>"Imported Classes:"</do-it>

<do-it>self error: 'Attempting to file-in parcel imports.  Choose terminate or close'</do-it>

<class>
<name>MimeEntity</name>
<environment>Net</environment>
<super>Net.MessageElement</super>
<private>false</private>
<indexed-type>none</indexed-type>
<inst-vars>header headerCharset </inst-vars>
<class-inst-vars>builderClass printerClass writerClass </class-inst-vars>
<imports></imports>
<category>Net-MIME-Support</category>
<attributes>
<package>MIME</package>
</attributes>
</class>

<class>
<name>HttpRequest</name>
<environment>Net</environment>
<super>Net.HttpEntity</super>
<private>false</private>
<indexed-type>none</indexed-type>
<inst-vars>requestLine </inst-vars>
<class-inst-vars></class-inst-vars>
<imports></imports>
<category>Net-HTTP-Support</category>
<attributes>
<package>HTTP</package>
</attributes>
</class>

<class>
<name>HttpClient</name>
<environment>Net</environment>
<super>Net.NetClient</super>
<private>false</private>
<indexed-type>none</indexed-type>
<inst-vars>request keepAlive protocol </inst-vars>
<class-inst-vars></class-inst-vars>
<imports></imports>
<category>Net-HTTP-Support</category>
<attributes>
<package>HTTP</package>
</attributes>
</class>

</st-source>
