Tag Archives: PHP

CSV file upload with ZIP containers, jQuery, Ajax and PHP 5.4 progress tracking – VI

Posted on by

In the previous articles of this series about an Ajax controlled file upload with PHP progress tracking

CSV file upload with ZIP containers, jQuery, Ajax and PHP 5.4 progress tracking – V
CSV file upload with ZIP containers, jQuery, Ajax and PHP 5.4 progress tracking – IV
CSV file upload with ZIP containers, jQuery, Ajax and PHP 5.4 progress tracking – III
CSV file upload with ZIP containers, jQuery, Ajax and PHP 5.4 progress tracking – II
CSV file upload with ZIP containers, jQuery, Ajax and PHP 5.4 progress tracking – I

we have shown how we can measure and display the progress of a file upload process with a series of Ajax controlled polling jobs and the progress tracking features of PHP > 5.4. At least in our test example this worked perfectly.

However, for practical purposes and especially when our server users deal with large files we must in addition take better care of some limiting PHP parameters on the server. Both a good server admin and a program developer would, of course, try to find out what file sizes are to be expected on a regular basis and adjust the server parameters accordingly. However, you never know what our beloved users may do. What happens if we talked about file sizes of less than 100MB and suddenly a file with 200MB is transferred to the server?

For which limiting PHP parameters on the server may we run into serious trouble?

Due to security considerations the PHP module interaction with incoming data streams is limited by parameters set for the Apache server. The relevant configuration file is e.g. on Opensuse located at

/etc/php5/apache2/php.ini

The most important limits (set in different sections of the file) are:

; Maximum amount of time each script may spend parsing request data. It’s a good
; idea to limit this time on productions servers in order to eliminate unexpectedly
; long running scripts.
; Default Value: -1 (Unlimited)
; Production Value: 60 (60 seconds)
; http://php.net/max-input-time
max_input_time = 200
 
; Maximum size of POST data that PHP will accept.
; Its value may be 0 to disable the limit. It is ignored if POST data reading
; is disabled through enable_post_data_reading.
; http://php.net/post-max-size
post_max_size = 200M
 
; Maximum allowed size for uploaded files.
; http://php.net/upload-max-filesize
upload_max_filesize =150M
 
; Maximum amount of memory a script may consume (128MB)
; http://php.net/memory-limit
memory_limit = 500M

The description of the first parameter above is somewhat unclear. What is meant by the “time spent on parsing request data”? Is this a part of the (also limited) execution time of the PHP target program of our Ajax transaction? Or is this limit imposed on the time required to read incoming POST data and to fill the $_POST array? If the latter were true a small bandwidth could lead to a violation of the “max_input_time” limit …

Regarding the second parameter the question turns up, whether this
limit is imposed on all transferred POST data including the file data?

The third parameter seems to speak for itself. There is a limit for the size of a file that can be transmitted to the server. However, it is not clear how this parameter affects real world scenarios. Does it stop a transfer already before it starts or only when the limit is reached during the transfer?

Regarding the 4th parameter we may suspect that it becomes important already during the handling (reading, parsing) of the incoming POST data. So, how much of memory (RAM) do we need at the server to handle large files during an upload process?

Warning regarding PHP parameter changes for multi-user situation on real world servers

We were and are discussing a privileged situation in this article series: Only one user uploads exactly one big Zip-container file to a server.

In such a situation it is relatively safe to fiddle around with PHP parameters of the central “php.ini” file (or PHP parameter settings in directory specific files; see the last section of this article). However, as an administrator of a server you should always be aware of the consequences of PHP parameter changes, e.g for memory limits, in a multi-user environment.

In addition you must also take into account that our code examples may be extended towards the case that one user may upload multiple files in parallel in one Ajax transaction.

Remarks on “max_input_time” – you can probably ignore it!

If you look up information about “max_input_time” available on the Internet you may experience that some confusion over the implications of this parameter remains. Especially as PHP’s own documentation is a bit contradictory – just compare what is said in the following manual pages:

 
Therefore, I tested a bit with files up to 1 GByte over slow and fast connections to PHP servers on the Internet. I came to the conclusion that the answer in the following “stackoverflow” discussion
http://stackoverflow.com/questions/11387113/php-file-upload-affected-or-not-by-max-input-time
describes the server behavior correctly. This means:

This parameter has no consequences with respect to connection bandwidth and the resulting upload time required for file data: It does not limit the required upload time. Neither does it reduce the amount of allowed maximum execution time of the PHP program triggered at the end of the file transfer process to the server.

“max_input_time” imposes a limit on the time to read/parse the data

  • after they have completely arrived at the server
  • and before the PHP program, which shall work with the data, is started.

This “parsing” time normally is very small and the standard value of 60 secs should be enough under most circumstances. If these findings are true we do not need to care much about this parameter during our file transfer process to the server. A value of 60 secs should work even for large files of 1 GB ore more on modern servers. At least for a server with sufficient resources under average load.

See also:

 
However, I can imagine circumstances on a server with many users under heavy load, for which this parameter nevertheless needs to be adjusted.

What does the PHP documentation say about the parameters “post_max_size”, “upload_max_filesize” and “memory_limit”?

Regarding these parameters we get at least some clear – though disputable – recommendation from the PHP documentation. At

 
we find the following explanation for “post_max_size”:

Sets max size of post data allowed. This setting also affects file upload. To upload large files, this value must be larger than upload_max_filesize. If memory limit is enabled by your configure script, memory_limit also affects file uploading. Generally speaking, memory_limit should be larger than post_max_size. When an integer is used, the value is measured in bytes. Shorthand notation, as described in this FAQ, may also be used. If the size of post data is greater than post_max_size, the $_POST and $_FILES superglobals are empty. This can be tracked in various ways, e.g. by passing the $_GET variable to the script processing the data, i.e. <form action=”edit.php?processed=1″>, and then checking if $_GET[‘processed’] is set.

Off topic: For those who find the track-recommendation in the last sentence confusing as it refers to $_GET, see e.g.

 
You can add parameters to your URL and these parameters will appear in $_GET, but if you decided to use the POST mechanism for data transfer these URL-parameters are included in the POST data mechanism of HTTP.

The recommendation for memory sizing is misleading in case of file uploads!

Following the recommendation quoted above would lead to the following relation for the PHP setup:

memory_limit > post_max_size > ( upload_max_file_size * number of files uploaded in parallel ).

Regarding the right side: My understanding is that “upload_max_file_size” sets a limit for each individual file during an upload process. See

 
Actually, I find the recommendation for the parameter “memory_limit” very strange. This would mean that somebody who has to deal with an upload file with a size of 2 GByte would have to allow for memory allocation for a single PHP process in the RAM > 2 GByte. Shall we take such a requirement seriously?

My
answer is NO ! But, of course, you should always test yourself ….

To me only the last relation on the right side of the relation chain makes sense during an upload process. Of course PHP needs some RAM and during file uploads also buffering requires sufficient server RAM. But several GByte to control a continuous incoming stream of data which shall be saved as a file into a directory (for temporary files) on the server? No way! I did some tests – e.g. limit the memory to 32 MB and successfully upload a 1 GB file. Therefore, I agree completely with the findings in the following article:

 
See also:

 
So:

Despite you need RAM for buffering during file uploads it is NOT required to use as much physical RAM as the size of the file you want to upload.

However, it may be wise to have as much RAM as possible if you intend to operate on the file as a whole. This may e.g. become important during phases when a PHP program wants to rewrite file data or read them as fast as possible for whatever purpose. A typical example where you may need sufficient memory is image manipulation.

Nevertheless: Regarding the file transfer process to the server itself the quoted recommendation is in my opinion really misleading. And: Do not forget that a high value for “memory_limit” may lead to server problems in a multi-user situation.

“post_max_size” and “upload_max_filesize” as the main limiting PHP parameters for file uploads

So, only the following condition remains:

post_max_size > upload_max_file_size * number of files uploaded in parallel

But this condition should be taken seriously! There are several things that need to be said about these parameters.

  1. A quick test shows: “post_max_size” imposes a limit on all POST data transferred from client – including file data.
  2. Even for situations in which only one file is uploaded I personally would choose “post_max_size” to be several MBs bigger than “upload_max_filesize”. Just to account for overhead.
  3. In case of an upload of multiple files in parallel (i.e. a situation, which we have not studied in this article series) you have to get an idea about the typical size and number of files to be uploaded in parallel. In such a situation you may also want to adjust the parameter

    ; Maximum number of files that can be uploaded via a single request
    max_file_uploads = 20

  4. There may be differences depending on the PHP version of how and when the server reacts to a violation of either of both parameters. For PHP 5.4 it seems that the server does not allow for an upload if either of the parameters is violated by the size of the transferred file(s) – meaning: the upload does not even start. This in turn may lead to different error situations on the server and
    messages issued by the server – depending on which parameter was violated.
  5. From a developer’s perspective it is a bit annoying that the PHP servers reaction to a violation of “upload_max_filesize” is indeed very different from its reaction a violation of “post_max_size”. See below.

Server reactions to violations of “post_max_size” and “upload_max_filesize”

We need to discuss a bit the reactions of a PHP server towards a violation of the named parameters before we can decide how to react within our PHP or Javascript programs in the course of an Ajax transaction.

Server reaction to a violation of “upload_max_filesize”
The Apache/PHP server reacts to a violation of “upload_max_filesize” by a clear message in

$_FILES[‘userfile’][“error”]

where ‘userfile’ corresponds to the “name” attribute of the HTML file input element. A reasonable way how to react to PHP error messages in $_FILES by PHP applications is described in the highest ranked comment of
http://php.net/manual/en/features.file-upload.errors.php
and also here
https://blog.hqcodeshop.fi/archives/185-PHP-large-file-uploads.html

Server reaction to a violation of “post_max_size”
What about a violation of “post_max_size”? We can only react reliably to an error via our PHP target programs if an error number or a clear, structured message is provided. Unfortunately, this is not the case when the sum of uploaded data via POST becomes bigger than “post_max_size”. When the server detects the violation no content at all is made available in $_POST or $_FILES. So, we have no error-message there a PHP program could react to.

However, we can combine

  • a test for emptiness of the superglobals $POST and $_FILES
  • with some HTTP information from the client, which is saved in $_SERVER,

to react properly in our PHP programs. Such a reaction within our Ajax transactions would naturally include

  • the creation of an error code and an error-message
  • and sending both back within the JSON response to the Javascript client for error control.

When we make a POST request to the server a value of the POST content size is provided by the client and available via the variable

$_SERVER[‘CONTENT_LENGTH’].

See:

 
So, for the purpose of error control we will need to add some test code to the “initial” PHP target program “handle_uploaded_init_files.php5” of our Ajax transaction which started the file upload.

Reasonable reactions of our PHP upload and polling programs to a violation of “post_max_size”

Remember that our initial Ajax transaction for upload triggered the server file “handle_uploaded_init_files.php5”. Therefore, we should some additional code that investigates the violation of post_max_size” there. This would probably look similar to: 

if (
	isset( $_SERVER['REQUEST_METHOD'] )      &&
        ($_SERVER['REQUEST_METHOD'] === 'POST' ) &&
        isset( $_SERVER['CONTENT_LENGTH'] )      &&
        (
 empty( $_POST ) )
 ) {
	$max_post_size = ini_get('post_max_size');
	$content_length = $_SERVER['CONTENT_LENGTH'] / 1024 / 1024;
	if ($content_length > $max_post_size ) {
		....
		// Our error treatment ....
		$err_code = ....;
		// create an error message and send it to the Ajax client 
		$err_post_size_msg = ".....";
		....
	}

	....
	....
	// transfer the error code and error message to some named element of the JSON object 
	....
	$ajax_response['err_code'] = $err_code;
	$ajax_response['err_msg'] = $err_post_size_msg;
	.....
	$response = json_encode($ajax_response);
	echo $response;
	exit;
}

 
See also:

&nbsP;
Note that we cannot assume a certain timing of the reaction of the main program in comparison to our polling jobs. It may happen that we have already started the polling sequence before the error messages from our first Ajax transaction arrive at the client. Therefore, also our polling jobs “check_progress.php5” should be able to react to empty superglobals $_POST and $_FILES : 

if ( ( empty( $_POST ) ) && empty ( $_FILES ) ) {
	// Our error treatment ....
	// create an error message and send it to the Ajax client 
	// refer to messages that may turn up in parallel from the main PHP program
	....
}

 
The different Javascript client methods which receive their respective Ajaj messages should evaluate the error messages and error numbers from the server, display them and, of course, stop the polling loop in case it is still active. As these are trivial programming steps we do not look deeper into them.

Avoid trouble with limiting PHP parameters before starting the file upload

Although we can react to error situations as described above I think it is better to avoid them. Therefore, I suggest to check file size limits before starting any upload process.

In our special situation with just one big Zip-file to upload we can initiate a file size limit check on the server as soon as we choose the file on the client. This means that the Javascript client must be enabled to react to the file selection action and request some information about the parameters “post_max_size” and “upload_max_filesize” from the server. In addition we need a method to compare the server limits with the size of the chosen file.

Looking into
CSV file upload with ZIP containers, jQuery, Ajax and PHP 5.4 progress control – II
we see that we had defined a proper Javascript Control Object [CtrlO] for the upload form 

<form id="form_upload" name="init_file_form"  action="handle_uploaded_init_files.php5" method="POST" enctype="multipart/form-data" >

 
which – among other things – contains the file selection input tag:

<input type="
file" name="init_file" id="inp_upl_file" >

 
However, we had not assigned any method to the file selection process itself. We are changing this now:

function Ctrl_File_Upl(my_name) {
	
	this.obj_name = "Obj_" + my_name; 
	
	// Controls related to GOC and dispatched object addresses
	this.GOC = GOC;
        this.SO_Tbl_Info = null; 
        this.SO_Msg      = null; 
			
	// ay to keep the selected file handles 
	this.ay_files = new Array(); 
		
	// msg for 1st Ajax phasefor file upload; 
	this.msg1 = ''; 
		
	// Timeout for file transfer process
	// this.timeout = 500000; // internet servers  
	this.timeout = 300000; 
		
	// define selectors (form, divs) 
	this.div_upload_cont_sel 	= "#" + "div_upload_cont";
	this.div_upload_sel 		= "#" + "div_upload";
	this.p_header_upload_sel 	= "#" + "upl_header" + " > span";
			
	this.form_upload_sel 		= "#" + "form_upload";
	this.input_file_sel 		= "#" + "inp_upl_file";
	this.upl_submit_but 		= "#" + "but_submit_upl";

	this.hinp_upl_tbl_num_sel	= "#" + "hinp_upl_tbl_num";			
	this.hinp_upl_tbl_name_sel	= "#" + "hinp_upl_tbl_name";			
	this.hinp_upl_tbl_snr_sel	= "#" + "hinp_upl_tbl_snr";			
	this.hinp_upl_succ_sel 		= "#" + "hinp_upl_succ";			
	this.hinp_upl_run_type_sel 	= "#" + "hinp_upl_run_type";			
	this.hinp_upl_file_name_sel 	= "#" + "hinp_upl_file_name";			
	this.hinp_upl_file_pipe_sel 	= "#" + "hinp_upl_file_pipe";

	// display the number of extracted and processed files 
	this.num_open_files_sel		= '#' + "num_open_files";
	this.num_extracted_files_sel 	= '#' + "num_extracted_files";
	
	// Other objects on the web page - progress area 
	this.trf_msg_cont	= '#' + "trf_msg_cont";
	this.trf_msg		= '#' + "trf_msg";
	this.imp_msg_cont	= '#' + "imp_msg_cont";
	this.imp_msg		= '#' + "imp_msg";
				
	// Status (!) message box (not the right msg box) 
	this.status_div_cont = '#' + "status_div_cont";
	this.id_progr_msg_p  = "#progr_msg"; 
	
	//progress bar 
	this.id_bar = "#bar"; 
			
	// right msg block
	this.span_main_msg	= "span_msg"; 
	
	// variables to control the obligatory check of the file size 
	this.file_size_is_ok = 1; 
			
	// variables for the Ajax response 
	this.upl_file_succ 	= 0; 
	this.upl_file_name 	= ''; 
        
	// File associated variables
	this.file_name      = '';
        this.file_size_js   = 0; 	// file size detected by JS
        this.file_size      = '';	// file size detected by server
        this.allowed_file_size = 0;	// allowed file size for uploads on the server  
        
	// Processing of files     
        this.num_extracted_files = 0;
	this.file_pipeline 	 = 0; 
	this.import_time 	 = 0; 
	this.transfer_time 	 = 0; 
        
	this.name_succ_dir = ''; 
	this.num_open_files = 1;  

	// transfer time measurement 
	this.date_start = 0;   
	this.date_end = 0; 
	this.ajax_transfer_start = 0; 
	this.ajax_transfer_end 	 = 0; 
			
	// database import time measurement 
	this.date_data_import_start = 0; 
	this.date_data_import_end = 0; 
	this.data_import_start = 0; 
	this.data_import_end = 0; 
			
	this.transfer_time	= 0; 
	this.processing_time = 0; 
	this.time_start = 0; 
			
	// Determine URL for the Form 
	this.url = $(this.form_upload_sel).attr('action'); 
	console.log("Form_Upload_file - url = " + this.url);  				

	// Register methods for event handling 
	this.register_form_events(); 
			 
}	
	
// Method to start uploading the file  
// -------------------------------------------------------------------
Ctrl_File_Upl.prototype.register_form_events = function() {
			
	$(this.input_file_sel).
click(
		$.proxy(this, 'select_file') 
	);
			
	$(this.input_file_sel).change(
		$.proxy(this, 'fetch_allowed_file_size') 
	);
	
	$(this.upl_submit_but).click(
		$.proxy(this, 'submit_form') 
	);
				
	$(this.form_upload_sel).submit( 
		$.proxy(this, 'upl_file') 
	); 
					
};

 
The reader recognizes that in contrast to the version of the CtrlO “Ctrl_File_Upl” discussed in previous articles of this series we have added some selector IDs for fields of some other web page areas. But the really important change is an extension of the methods for additional events in “Ctrl_File_Upl.prototype.register_form_events()”:

First, we react to a click of the file selection button of the file input field. This only serves the purpose of resetting fields and message areas on the web page. But we react also to the file selection itself by using the change event of the file input field. This triggers a method “fetch_allowed_file_size()” which retrieves the parameter “upload_max_filesize” from the server.

Note:
We assume here that the server admin was clever enough to set post_max_size > upload_max_filesize!
Therefore, we only will perform a file size comparison with the value of “upload_max_filesize”. If you do not trust your server admin just extend the methods and programs presented below by an additional and separate check for file sizes bigger than “post_max_size”. This should be an easy exercise for you.

Now, let us have a look at the new methods of our Javascript CtrlO : 

	
// Method to react to a click on the file selection box 
// ---------------------------------------------------- 
Ctrl_File_Upl.prototype.select_file = function (e) {
	// Call method to reset information and message fields 
	// Note: The following method also deactivates the file submit button !  
	this.reset_upl_info(); 
};

	
// Method to check whether file size is too big   
// ---------------------------------------------
// We check whether the file size is too big 
Ctrl_File_Upl.prototype.fetch_allowed_file_size = function (e) {
			
	this.file_size_is_ok = 0; 
			
	// size of the file im MByte determined on the client 
	this.file_size_js = $(this.input_file_sel)[0].files[0].size/1024/1024;
	console.log("actual file size of chosen file = " + this.file_size_js); 	
			
	// Now trigger an Ajaj transaction 
	var ajax_url = "../func/get_allowed_file_size.php5"; 
	var form_data = ''; 
			
	// 03.07.2015: we avoid setup as this would be taken as the standard for subsequent Ajax jobs 
	$.ajax({
                //contentType: "application/x-www-form-urlencoded; charset=ISO-8859-1",
                // context:  Ctrl_Status
                url: ajax_url, 
		context:  this, 
		data: form_data, 
		type: 'POST', 
		dataType: 'json', 
                success: this.response_allowed_file_size, 
                error: this.error_allowed_file_size
        });
};

// Method for Ajaj error handling during file size check transaction 		
// --------------------------------------------------------------	
Ctrl_File_Upl.prototype.error_allowed_file_size = function(jqxhr, error_type) {
			
	// Reset the cursor 
	$('body').css('cursor', 'default' ); 

	// Error handling
	console.log("From Ctrl_File_Upl ::  got Ajax error for fetch_allowed_file_size" );  
	var status = jqxhr.status; 
	var status_txt = jqxhr.statusText; 
	console.log("From Ctrl_File_Upl.prototype.error_allowed_file_size() ::  status = " + status );  
	console.log("From Ctrl_File_Upl.prototype.allowed_file_size ::  status_text = " + status_txt ); 
	console.log("From Ctrl_File_Upl.prototype.allowed_file_size ::  error_type = " + error_type ); 
			
	var 
msg = "<br>Status: " + status + "  Status text: " + status_txt;    
	this.SO_Msg.show_msg(1, msg); 
};

// Method for Ajaj rsponse handling after file size check transaction 		
// --------------------------------------------------------------			
Ctrl_File_Upl.prototype.response_allowed_file_size = function (json_response, success_code, jqxhr) {
	
	// Reset the cursor 
	$('body').css('cursor', 'default' ); 
				
	var new_msg; 
	var status = jqxhr.status; 
	var status_txt = jqxhr.statusText; 
	console.log("response_allowed_fsize: status = " + status + " , status_text = " + status_txt ); 	

	// The allowed file size on the server
	this.allowed_file_size = json_response['allowed_size'];	
	// parseInt required due to possible MB or GB endings on the server 
	this.allowed_file_size = parseInt(this.allowed_file_size); 
	console.log("allowed file size on server = " + this.allowed_file_size); 	

	// size comparison
	// ----------------
	if ( this.file_size_js > this.allowed_file_size ) {
		this.file_size_is_ok = 0; 
		new_msg = $(this.span_main_msg).html();
		if (new_msg == undefined) {
			new_msg = ""; 
		}
		new_msg += "<br><span style=\"color:#A90000;\">File size too big.</span><br>" +
		"The server allows for files with a size ≤ " +  	
		parseFloat(this.allowed_file_size).toFixed(2) + " MB." + "<br>" + 
		"The size of the chosen file is " + parseFloat(this.file_size_js).toFixed(2) + " MB." + "<br><br>" + 
		"<span style=\"color:#A90000;\">Please choose a different file or reduce the contents !</span>" + "<br><br>" + 
		"If you permanently need a bigger file size limit on the server, please contact your administrator"; 
					
		this.SO_Msg.show_msg(0, new_msg); 
				
	// file size within limits 
	// -------------------------
	else {
		this.file_size_is_ok = 1; 
					
		new_msg = $(this.span_main_msg).html();
		if (new_msg == undefined) {
			new_msg = ""; 
		}
		new_msg += "<br><span style=\"color:#007700;\">File size within server limits.</span><br>" +
		"The server allows for files with a size ≤ " +  parseFloat(this.allowed_file_size).toFixed(2) + " MB." + "<br>" + 
		"The size of the chosen file is " + parseFloat(this.file_size_js).toFixed(2) + " MB." + "<br><br>" + 
		"<span style=\"color:#007700;\">Use the "Start Upload" button to start the file upload!</span>"; 
					
		this.SO_Msg.show_msg(0, new_msg); 
					
		// reactivate the submit button 
		// -----------------------------
		$(this.upl_submit_but).on("click", $.proxy(this, 'submit_form') ); 
		$(this.upl_submit_but).css("color", "#990000"); 
	}
};
	
// Method to reset some form and information fields on the web page 
// ---------------------------------------------------------------
// We have to reset some form and message fields
Ctrl_File_Upl.prototype.reset_upl_info = function() {
				
	var msg_progr = ''; 
	$(this.id_progr_msg_p).html('');
				
	var msg_trf = ''; 
	$(this.trf_msg_cont).css('display', 'none');
	$(this.trf_msg).css('color: #666'); 
	$(this.trf_msg).html(msg_trf);
		    	
	var msg_imp = ''; 
	$(this.imp_msg_cont).css('display', 'none'); 
	$(this.imp_msg).css('color: #666'); 
	$(this.imp_msg).html(msg_imp); 
		    	
	// Deactivate the "Start Upload" Button 
	// ------------------------------------
	$(this.upl_submit_but).off("click"); 
	$(this.upl_submit_but).css("color", "#BBB"); 

	// Reset also the main message area  
	// ----------------------------------------------
	this.SO_Msg.show_msg(0, ''); 

};

 
This is all pretty straightforward and parts of it are already well known of our
previous descriptions for handling the Ajaj interactions with the server by the help of jQuery functionality.

A short description of what happens is:

  • When you click on the button of the file selection input field contents of fields in the message area of our web page and information fields about upload progress are reset as we assume that a new upload will be started.
  • During reset also the form’s submit button to start a file upload via Ajax/Ajaj is disabled. Note that we use jQuery’s “off(‘event’)”-functionality to to this.
  • As soon as the user selects a specific file we trigger a method which determines and saves the size of the chosen file to a variable and starts an Ajax transaction afterwards. This Ajax interaction calls a target PHP program “get_allowed_file_size.php5” in some directory.
  • The JSON-response of the PHP program is handled by the method
    Ctrl_File_Upl.prototype.response_allowed_file_size.

      The main purpose of this method is to make a comparison of the already determined file size with the limit set on the server and issue some warnings or positive messages. If the file size of the chosen file is within the server’s limit we reactivate our “submit” button of the upload form. (Note that we use jQuery’s “on(‘event’)”-functionality to to this.) Otherwise we keep it inactive – until a more suitable file is chosen by the user.

Thus, by very simple means we prevent any unreasonable upload process already before it can be started by the user.

It remains to show an excerpt of the simple PHP target file: 

<?php

// start session and output buffer
session_start();
ob_start(); 

$file_size_limit = ini_get("upload_max_filesize");
$ajax_response = array();
$ajax_response['allowed_size'] = $file_size_limit; 

$ajax_response['sys_msg'] .= ob_get_contents();
ob_end_clean();

$response = json_encode($ajax_response);
echo $response;
exit; 

?>

Nothing special to discuss here.

Can we change the limiting parameters during PHP program execution?

No, we can not. But as a developer you may be able to define directory specific settings both for “post_max_size” and “upload_max_filesize” on the server by uploading “.htaccess”-files or “.user.ini”-files to program directories – if this is allowed by the administrator.

The web page php.net/manual/en/ini.core.php shows a column “Changeable” for all important parameters and the respective allowed change mechanisms.
See also:
http://php.net/manual/en/ini.list.php

Different methods of how to change PHP parameters as a user are described here:

 
However, if you are not a developer but a server admin, preventing users from changing PHP ini-paramters may even be more important for you:

 
Enough for today. In the next article of this series

CSV file upload with ZIP containers, jQuery, Ajax and PHP 5.4 progress tracking – VII

we shall have a look at possible problems resulting from timeout limits set for our Ajax transactions.

PHP Code Content Assists und Inline Type Hinting in Eclipse

Posted on by

Vor ein paar Jahren war ich sehr happy, als ich entdeckte, dass Eclipse mit dem PHP Pluging (PDT) etliche Möglichkeiten im Bereich des PHP Content Assistings bietet. So nutze ich die Möglichkeiten des Type Hintings mittels “phpDocumentor tags” ausgiebig oberhalb der Deklaration von Class Member Variablen oder bei der Definition von Funktionen (frei oder als Objekt-Methoden). Es ist einfach schön, bequem und sehr effizient, wenn “Ctrl-Space” das Tippen von neuem Code in einem PHP-Projekt mit passenden Vorschlägen aufgrund der Definition von Klassen etc. in anderen Dateien unterstützt. Fast magisch !

Was ich bisher nicht hinbekommen hatte, war das Type Hinting und “Content Assisting” zu Variablen im PHP-Code, die im Rahmen einer dynamischen Namensgebung über die Nutzung der Funktion “call_user_func” als Objekt einer Klasse instanziiert werden. Das braucht man z.B., wenn man das Singleton-Pattern benutzt und das Singleton-Objekt entweder erstmalig instanziiert werden muss oder aber die Referenz auf das ggf. schon existierende Singleton-Objekt geholt werden soll – soweit es bereits woanders instanziiert wurde.

Aber auch das geht unter Eclipse PDT mit sog. “Inline Type Hinting”. Ein Beispiel: 

	$get_instance ="getInstance"; 
	$Class_SS_Run_Data_Name = "Class_SS_Run_Data";
	$ay_SS_Run_Data		= array($Class_SS_Run_Data_Name, $get_instance);

	/* @var $SS_Run_Data Class_SS_Run_Data */		
	$SS_Run_Data = call_user_func($ay_SS_Run_Data);
	unset($ay_SS_Run_Data);

getInstance ist hier eine statische Methode, die gemäß des Singleton Patterns entweder das Objekt instanziiert oder die Referenz auf das Singleton Objekt aus einer statischen Variablen ermittelt.

Wichtig ist hier aber die Kommentar-Zeile und ihr Aufbau. Die Klasse “Class_SS_Run_Data” ist in einer anderen Datei definiert, deren Source Pfad natürlich im Bereich des BUILD Path des PHP-Projektes liegen muss.

Tippe ich nun später “$SS_Run_Data->” im Code erhalte ich je nach Einstellungen des Content Assistings umgehend oder spätestens über Ctrl-Space eine Vorschlagsliste zu Variablen oder Methoden der Klasse. Super !

Gelernt habe ich das nach Lesen eines Artikels von Norm McGarry, der sich mal die Mühe gemacht hat, die verschiedenen Arten des “Type Hintings” für “Content Assist”-Zwecke unter Eclipse PDT zusammenzustellen. Siehe:
PHP Type Hinting with Eclipse

Herzlichen Dank an Norm McGarry!

PHP/OO/Schemata: Composite SLAVE objects and (X)HTML generator methods

Posted on by

Some time ago I wrote an article about how to treat groups of associated, related properties of a Real World Object [“RWO”] when representing it by an object [“PWO”] in PHP.

We assumed that the property description of a certain object class is done with the help of a so called “SCHEMA” (a file with a list of definitions and rules or Schema program using definitions saved in a database Schema table). A SCHEMA is specific for a PWO class. A SCHEMA defines object relations, object properties, the relations between certain object properties, the relation of object properties with database fields and of course the type of each property, the property’s own properties as well as associated constraints. Such “SCHEMATA” would play a central role in a web application as they encapsulate vital structural and detail information about objects and their properties.

Every object constructors would refer to an appropriate SCHEMA, as well as web generators for the creation e.g. of form or web template elements would do. Actually, any reasonable (web) application would of course work with multiple SCHEMATA – each for a different object class. SCHEMATA give us a flexible mode to change or adapt a classes properties and relations. They also close the gap between the OO definitions and a relational database.

To handle groups of associated properties in such an environment I suggested the following:

  • Split the SCHEMA-information for properties and related database fields into several “SLAVE-Schemata” which would be built as sub objects within the MAIN SCHEMA object. Each SCHEMA would describe a certain group of closely associated object properties.
  • Create and use “SLAVE PWO SGL objects” as encapsulated sub-objects in a MAIN SGL PWO. Each SLAVE PWO object gets it’s own properties defined in a related SLAVE-SCHEMA.
  • Each of the SLAVE PWO objects receives it’s knowledge about it’s individual SLAVE-SCHEMA by an injection process during construction.

See:
PHP/OO/Schemata: Decouple property groups by combined composite patterns

Each PWO object, representing a single RWO we shall call a SGL PWO. It comprises a series of sub-objects: SLAVE SGL PWOs that refer to corresponding SLAVE Schemata of a MAIN Schema. See the illustration from the article named above:

Slave_schemata

In the named article I had discussed that one can iterate methods for complete database interactions over the MAIN and the SLAVE objects. The same is true for interactions with POST or SESSION data in a CMS like web application. So, there is no need to rewrite core method code originally written for objects which comprise all object properties in just one property/field-SCHEMA without any SLAVE objects. At least with respect to field checks and database interactions.

This works if and when a SGL object is derived from a base class which comprises all required methods to deal with the database, POST and SESSION data. Each MAIN or SLAVE SGL PWO knows itself how to store/fetch it’s property data into/from database tables (or $_POST or $_SESSION arrays) and uses the same methods inherited from common SGL base classes (= parent classes) to do so.

I have meanwhile introduced and realized this SLAVE PWO and SLAVE SCHEMA approach in a quite general form in my web application framework. In this article I briefly want to discuss some unexpected consequences for HTML
generator methods in a web or CMS context.

(X)HTML-Generator methods – where to place them ?

When you design CMS like web applications you always have to deal with Template [TPL] structures and objects that fill the templates with reasonable formatted contents. The contents may be just text or images or links inserted into template placeholders. In more complicated cases (like e.g. the generation of maintenance masks with forms), however, you may have to generate complete HTML fragments which depend on property/field information given in your respective SCHEMA.

A basic design question is: Where do we place the generator methods? Should the SGL PWOs know how to generate their property contents or should rather a “Template Control Object” – a “TCO” – know what to do? I have always preferred the second approach. Reason:

TPL aspects may become very specific, so the methods may need to know something about the TPL structures – and this is nothing that I want to incorporate into the OO representation [PWO] of real world objects.

Over time I have developed a bunch of generator methods for all kind of funny things required in customer projects. The methods are defined in base classes for Template Control Objects and or special purpose sub classes injected into TCOs. A TCO knows about its type of TPL and works accordingly. (By the way: With Pear ITX or Smarty you can realize a complete separation of (X)HTML-code and the PHP code).

(X)HTML-Generator methods – which MAIN or SLAVE PWO and which MAIN/SLAVE SCHEMA are we dealing with ?

In addition to some property/field identifiers a (X)HTML generator method has of course to know what SGL PWO and what SCHEMA it has to work with. This information can be fetched either by the TCO due to some rules or can be directly injected into the methods.

In the past I wanted to keep interfaces lean. In many applications the SGL PWO object was a classical singleton. So, it could relatively easily be received or identified by the central TCO. I did not see any reason to clutter TCO method interfaces with object references that were already known to their TCO object. So, my generator methods referred and used the SGL PWO object and it’s SCHEMA by invoking it with the “$this”-operator:

function someTCOgenerator_method() {
…..
do something with $this->SGL and $this->Schema
….
}

However, in the light of a more general approach this appears to be a too simplistic idea.

If we regard a SLAVE SGL object as a relatively compact entity inside a PWO – as a SLAVE object with its own property and field information SCHEMA – than we see: for a generator method it behaves almost like an independent object different from the MAIN PWO. This situation is comparable to one where the generator method really would be requested to operate on instances a completely different PWO class:

A HTML generator method needs to know the qualities of certain OO properties and associated database field definitions. In our approach with MAIN SGL PWOs comprising composite SLAVE SGL PWO objects each SGL object knows exactly about its associated MAIN or SLAVE Schema object. To work correctly the generator method must get access to this specific SCHEMA. This would in a reasonable application design also be valid for PWO objects representing other, i.e. different types of RWOs.

A (X)HTML generator method can work properly as soon as it knows about

  • the SGL object,
  • the object property (identified by some name, index or other unique information) to operate on and generate HTML code for,
  • the SCHEMA describing the qualities of the object property and related database fields.

This would in
our approach also be given for our SLAVE SGL objects or any PWO as soon as we inject it into our generator method.

Therefore, (X)HTML generator methods of TCOs should be programmed according to the following rules:

  • Do not assume that there is only one defined class of PWO SGL objects that the Template Control Object TCO needs to know about and needs to apply it’s generator methods to.
  • Instead enable the (X)HTML generator methods of a TCO to be able to work with any kind of PWO and its properties – as long as the PWO provides the appropriate SCHEMA information.
  • Inject the SGL [SLAVE] PWO and thereby also its associated [SLAVE] SCHEMA into each TCO generator method:
    function someTCOgenerator_method($SGL_ext, …..).
  • Do not refer to the TCO’s knowledge about a SGL PWO by using the “$this” operator (like “$this->Sgl”) inside a generator method of a TCO; refer all generator action to a local $SGL object reference that points to the injected (!) object $SGL_ext:
    $SGL = $SGL_ext.
    Also refer to a local $Schema which points to the $SGL->Schema of the injected $SGL object:
    $Schema = $Sgl->Schema.
    (Remember: Objects are transferred by reference !)

These recipes give us the aspired flexibility to deal with properties both of SLAVE objects and objects of a different PWO class.

The injection is a small but structurally important difference in comparison to the database interaction methods directly incorporated in SGL objects or better their base classes. Here the TCO (!) method must receive the required information – and we gain flexibility when we inject the SGL object into it (plus information identifying the property for which a HTML fragment has to be generated for).

Had I respected the rules above already some time ago it would have saved me much time now. Now, I had to adapt the interfaces of many of my generator methods in my TCO base classes.

Again, I found one experience confirmed:

  • In case of doubt do not hesitate to use loose object coupling via injection on general methods which could in principle be applied to more and other objects than the ones the object containing the method knows about at the time of your class design.
  • Use injection even if it may look strange when you need to do something like
    $this->someTCOgenerator_method($this->KnownObject);
    i.e., when you inject something the present object containing the method knows about already.

It will save time afterwards when iterator patterns over other objects have to be used and when you may access the (public) method from outside, too.

Iteration over SLAVE objects

Now, if our (X)HTML-generator methods are prepared for injection of SGL PWO objects, we have no more difficulties to generate HTML fragments required in templates for selected properties of MAIN and SLAVE PWO SGL objects:

We just have to iterate the usage of the generator method for the properties/fields of the MAIN SGL PWO as well as its SLAVE SGL PWOs. By getting the SGL object injected the method also knows about the right SCHEMA to use and provide required detail information for the generation of related HTML code (e.g. for a form element).

Think about a situation in which we want to provide a form with input fields which a user can use to update data for all properties of a certain PWO. We just have to apply generator methods to create the required input field types according to the appropriate SCHEMA informations. For this

  • we loop (iterate) over the MAIN and all it’s SLAVE objects,
  • identify all properties according to the MAIN or SLAVE SCHEMA
    information
  • and apply the generator method by injecting the relevant property identifier plus the MAIN/SLAVE-SGL object in question.

Mission accomplished.

The principle of iteration over SLAVE objects is actually nothing new to us: We used it already when dealing with the database interaction methods. (It is a basic ingredient of a composite pattern).

If we only want to work on selected properties then we need to know which of the properties is located in which SLAVE PWO and described in which SLAVE SCHEMA. To be able to do so, we should create and use an array

  • that collects information about which PWO property belongs to which of the PWOs SLAVE objects
  • and which is filled in course of the construction process of a PWO.

Conclusion

When realizing a composite pattern in the form of SLAVE objects (with SLAVE Schemata) to deal with closely associated property groups of complex objects you can apply existing base class methods and iterate over the SLAVE objects to accomplish complete transactions affecting all properties of a structured PWO. This principle can be extended to (X)HTML generator methods of Template Control Objects, if these methods are prepared to receive the SLAVE SGL PWO objects by injection. If we only want to apply generator methods on a bunch of selected properties, we should use an array associating each PWO property with a SLAVE PWO SGL object.