.. _bonnespratiques: ================= Best practices ================= ----------- Principles ----------- Whether you want to develop a web application or a smartphone one, you will probably be concerned with its successful operation and availability. Here is some advice to allow you to get the best from our platform. ------------------------------------ Direct access to restricted datasets ------------------------------------ As you may have noticed, some of our data is only available if you already have an account and the authorization to use it. As soon as you implement these pieces of information in your code, it's up to you to take care of their safety, non-disclosure and their ease of use all through their life cycle (software updates, deletion...) In Javascript/Ajax ------------------- One of the peculiarities of Javascript is being fully loaded and executed on the client side. If you integrate your credentials in your javascript code, they will be automatically accessible to any user of your website. Using tricks and workarounds can certainly make their retrieval more difficult, but you can be sure that sooner or later they will appear in the request you will send towards our services. Thus, for any Javascript/Ajax development, you must use the `Proxyfication`_ techniques for all the requests described below. In PHP, Python, Java --------------------- No problem with these server-side languages for which you can follow the provided :ref:`examples` On mobile platform ---------------------- Applications developed on the mobile platform are quite safe, as they are generally compiled and are thus distributed to the users in a format which is really complicated to reverse-engineer and therefore retrieve the information written into it. On the other hand, you must consider a practical aspect tied to your credentials life cycle. Should you change your login, or even your account, how will the diffusion occur of the new credentials among all the devices using your software? Will the user have to update their software? Can you force this software to be mandatory? How many users are you ready to lose in such an operation? You must take this into account before starting to spread out your solution. Performance -------------- Another important question is about our platform performance. It is not sized to carry out the same workload as www.google.fr. If your application is successful, which we hope it will be, if a single tweet boosts it towards popularity summits very quickly, are you sure that our infrastructures will hold the sudden load and allow your users to have the best experience with your application? Conclusion -------------- For each of the below situations, you have to take care of the implementation of the recommended practices. The simplest way is sometimes to directly retrieve the data and serve it from your own premises. ----------------------------------- Data retrieval ----------------------------------- Your user rights allow you to retrieve the data and store it on your own servers. The retrieval frequency must be set according to their update frequency. Once implemented into your infrastructure, the resolution of the previously mentioned problems is far easier: * on mobile, the users direct access to your own version of the data. If your credentials change, it only affects your data retrieval script and not the application deployed on your users devices, which can use your own authentication/authorization system and procedures. * for performance : punctual retrieval, even frequent, has a minimal effect on our platform . Provided that your own infrastructure is correctly sized to hold the load generated by your application, you'll have no problem to fear on our side. ----------------------------------- Proxyfication ----------------------------------- Having the data installed in your infrastructure or directly accessed from your application to our services, there are use cases when the usage of proxyfication is however mandatory. It's mostly the case in a Javascript environment with c- with which you have two major concerns: * the impossibility of hiding the credentials to use for accessing the data. You can however have set, after having retrieved the data, a user specific authentication system, in which case you have already solved a part of the problem. * Access to a third-party domain in Ajax. This is strictly forbidden by the "Same Original Policy" which forces the ajax retrieved content to be located in the same domain and same port as the main web page from which the request has been emitted. This rule doesn't apply to images, which explains why we can have WMS external streams working perfectly well, but applies to all other types of services : WFS, JSON or KML. Please note that our servers are configured for allowing external domain connections through Ajax request, but if you rely on other data servers, you may face this kind of situation. In order to get around these restrictions, you can use the proxyfication trick. The basic principle is to send the Ajax requests to a script located on your own server (so having the same origin and port as your main page), which will transfer these requests to the external service, without suffering from the same constraints as the client. This can also be done to distinguish the credentials used to access your server from the ones used to access ours. WEB CLIENT------> PROXY SCRIPT ------> EXTERNAL SERVICE user:toto.........user:my_account......> Implementing a proxyfication script is rather trivial, but you have to respect a few rules. Don't leave it open to anybody for instance, or it will be used by malicious people to surf the internet anonymously. And don't let it do whatever the user asks it to either. Control the accepted operations and reject all others. Python implementation --------------------------- The python script used below is shipped with OpenLayers under the proxy.cgi name. This python script perfectly fits the needs for WFS usage. To activate it, you have to put it in an executable directory of the web server (generally cgi-bin suits this need), set its own rights to make it executable (i.e. chmod a+x proxy.cgi) and indicate its existence to OpenLayers for it to use it as a proxy (because OpenLayers is smart but not psychic !) with the directive : .. code-block:: javascript OpenLayers.ProxyHost = "/cgi-bin/proxy.cgi?url="; For any further information about the use of OpenLayers and a Proxy script, please refer to the dedicated FAQ : http://trac.osgeo.org/openlayers/wiki/FrequentlyAskedQuestions#ProxyHost .. code-block:: python #!/usr/local/bin/python """This is a blind proxy that we use to get around browser restrictions that prevents the Javascript from loading pages not on the same server as the Javascript. This has several problems: it's less efficient, it might break some sites, and it's a security risk because people can use this proxy to browse the web and possibly do unacceptable activities with it. It only loads pages via http and https, but it can load any content type. It supports GET and POST requests.""" import urllib2 import cgi import sys, os # Designed to prevent Open Proxy type stuff. # replace 'my_target_server' by the external domain you are aiming to allowedHosts = ['localhost','my_target_server'] method = os.environ["REQUEST_METHOD"] if method == "POST": qs = os.environ["QUERY_STRING"] d = cgi.parse_qs(qs) # checks if a url parameter exists in the POST request. If not, go to hell. if d.has_key("url"): url = d["url"][0] else: url = "http://www.openlayers.org" else: fs = cgi.FieldStorage() # checks if a url parameter exists in the GET request. If not, go to hell. url = fs.getvalue('url', "http://www.openlayers.org") try: host = url.split("/")[2] # reply with HTTP 502 code if the host is not allowed if allowedHosts and not host in allowedHosts: print "Status: 502 Bad Gateway" print "Content-Type: text/plain" print print "This proxy does not allow you to access that location (%s)." % (host,) print print os.environ # checks if the request is a http or https request elif url.startswith("http://") or url.startswith("https://"): if method == "POST": length = int(os.environ["CONTENT_LENGTH"]) headers = {"Content-Type": os.environ["CONTENT_TYPE"]} body = sys.stdin.read(length) r = urllib2.Request(url, body, headers) y = urllib2.urlopen(r) else: y = urllib2.urlopen(url) # print content type header i = y.info() if i.has_key("Content-Type"): print "Content-Type: %s" % (i["Content-Type"]) else: print "Content-Type: text/plain" print print y.read() y.close() else: print "Content-Type: text/plain" print print "Illegal request." except Exception, E: print "Status: 500 Unexpected Error" print "Content-Type: text/plain" print print "Some unexpected error occurred. Error text was:", E This PHP script does exactly the same : .. code-block:: php set_path($path); echo $result = $posting->send_request(); } // strip leading text from result and output result $len=strlen($result); $pos = strpos($result, "<"); if($pos > 1) { $result = substr($result, $pos, $len); } //$result = str_replace("xlink:","",$result); echo $result; // define class with functions to open socket and post XML // from http://www.phpbuilder.com/annotate/message.php3?id=1013274 by Richard Hundt class HTTP_Client { var $host; var $path; var $port; var $data; var $socket; var $errno; var $errstr; var $timeout; var $buf; var $result; var $agent_name = "MyAgent"; //Constructor, timeout 30s function HTTP_Client($host, $port, $data, $timeout = 30) { $this->host = $host; $this->port = $port; $this->data = $data; $this->timeout = $timeout; } //Opens a connection function connect() { $this->socket = fsockopen($this->host, $this->port, $this->errno, $this->errstr, $this->timeout ); if(!$this->socket) return false; else return true; } //Set the path function set_path($path) { $this->path = $path; } //Send request and clean up function send_request() { if(!$this->connect()) { return false; } else { $this->result = $this->request($this->data); return $this->result; } } function request($data) { $this->buf = ""; fwrite($this->socket, "POST $this->path HTTP/1.0\r\n". "Host:$this->host\r\n". "Basic: ".base64_encode("guillaume:catch22")."\r\n". "User-Agent: $this->agent_name\r\n". "Content-Type: application/xml\r\n". "Content-Length: ".strlen($data). "\r\n". "\r\n".$data. "\r\n" ); while(!feof($this->socket)) $this->buf .= fgets($this->socket, 2048); $this->close(); return $this->buf; } function close() { fclose($this->socket); } } function send_request() { global $onlineresource; $ch = curl_init(); $timeout = 5; // set to zero for no timeout // fix to allow HTTPS connections with incorrect certificates curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 1); //curl_setopt($ch, CURLOPT_USERPWD, 'guillaume:catch22'); //curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC); curl_setopt($ch, CURLOPT_URL,$onlineresource); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout); curl_setopt($ch, CURLOPT_ENCODING , "gzip, deflate"); if( ! $file_contents = curl_exec($ch)){ trigger_error(curl_error($ch)); } curl_close($ch); $lines = array(); $lines = explode("\n", $file_contents); if(!($response = $lines)) { echo "Unable to retrieve file '$service_request'"; } $response = implode("",$response); return utf8_decode($response); } ?> ------------- Opening hours ------------- Historically, the data published on data.grandlyon.com displays opening hours in a non-uniform way: * in the ‘openinghours’ attribute, using the `openinghours format `_, if present in the table * in the ‘openinghoursspecification’ attribute, using the `openinghoursspecification format `_, if present in the table To standardize and simplify the handling of opening hours, the Métropole has added a new attribute ‘horaires’ using the `OSM opening hours format `_. The Métropole encourages switching to the `OSM opening hours format `_ as soon as possible. The Métropole provides opening hours for both the current year (N) and the following year (N+1). The responsibility for determining whether a place is currently open or closed lies with the data consumer. Documentation ------------- Here are useful resources for understanding the OSM opening hours format: * `french version `_ * `english specification `_ (more complete) * `evaluation tool `_ * `entry and evaluation tool `_ Usage by the Métropole vs Official Specification ------------------------------------------------ To simplify the use of opening hours, the Métropole does not use the full capabilities of the `OSM opening hours format `_. Hours Display ~~~~~~~~~~~~~ Opening hours are presented as fragments, separated by semicolons. Each fragment corresponds to a period or a specific day. The fragments always appear in the following order: * default hours (openinghours) * specific date ranges (e.g., holidays or closure periods), listed chronologically * specific dates (e.g., public holidays or maintenance days), also listed chronologically In accordance with the format, each fragment overrides the previous ones. For instance, public holiday hours take precedence over default hours. Below is a reformulation with examples. The first fragment contains the default hours (e.g., ``Mo-Fr 09:00-12:00,14:00-17:00, Sa 09:00-17:00``). It includes no date and corresponds to the openinghours attribute. Chronologically listed specific date ranges follow (e.g., ``2023 Apr 01-2023 Oct 30 Mo-Fr 08:30-12:00,13:30-18:00, Sa 08:30-18:30``) Then come specific dates, also listed in chronological order: ``2023 Apr 10 closed "Easter Monday"; 2023 May 01 closed "Labour Day"`` Any fragment can include comments. Public holiday and school holiday names are automatically included as comments at the end of the fragment. Simplifying Considerations ~~~~~~~~~~~~~~~~~~~~~~~~~~ The `OSM opening hours format `_ allows for a wide range of schedule entries. The Métropole restricts its use to facilitate reuse and standardize the presentation. The description style follows this structure: | Allowed keywords are limited to: ‘24/7’ or ‘24/7 open’ (not ‘24/7 closed’, which will be represented by a missing value: horaires = NULL) The following are not used and will trigger a notification if detected: * values off and unknown * if is ‘24/7’, the ‘closed’ is not used — the schedule is omitted * in , only