GSM Drive test for experiencing customer point of view

GSM Drive test for experiencing customer point of view
Version 3.a

Table of Contents
A.    Benchmarking Product checking    3
Preliminary:    3
B.    Interoperator Benchmarking Methodology    4
1.    General    4
2.    Coverage benchmarking (idle mode)    5
3.    Voice benchmarking (Qos mode)    5
4.    PDD benchmarking (PDD mode)    5
5.    GPRS benchmarking (GPRS mode)    6
6.    SMS benchmarking (sms mode)    7
Country destination call    8



A.    Benchmarking Product checking
Preliminary:
i.    Team should invite regional to supervise before check activity begins.
ii.    The check product
a.    Done using T610 mobile station on a PC running on tems investigation must version 8.0 for 2G,
b.    Using K600 with tems investigation version 8.0 for 3G.
iii.    Check product test should be done at 5 specific place in a city with stationary mobile station in good radio condition for 35 times per simcard for 4 simcards:
a.    Indosat-Mentari
b.    Indosat-Im3 (510-01)
c.    Indosat-Im3 (510-21)
d.    Indosat-Matrix
iv.    If there is problem on the service that being tested, the team must inform indosat regional and headquarter immediately, in order to resolving the problem.
v.    The command sequence for MS in tools should be done in parallel, not allowed in serial sequence.
vi.    All 4 simcards will have to call a specific destination as a summarized in the following table:

√ = Test for 35 times
Location update test    Indosat
mentari    Indosat
Im3 (510 01)    Indosat
Im3 (510 21)    Indosat
matrix
Phone Number    0815-12345    0856-123456    0857-123456    0816-12456
Location update to 3G    √    √    √    √
Location update to 2G    √    √    √    √

√ = Test for 35 times
     Originating number
Destination number    Indosat    Indosat    Indosat    Indosat
    mentari    Im3 (510 01)    Im3 (510 21)    matrix
Phone Number    0815-12345    0856-123456    0857-123456    0816-12456
Voice test call                  
Indosat-mentari    √    √    √    √
Indosat-Im3 (510 01)    √    √    √    √
Indosat-Im3 (510 21)    √    √    √    √
Indosat-matrix    √    √    √    √
Fren    √    √    √    √
Telkomsel    √    √    √    √
Excelcomindo     √    √    √    √
Flexy    √    √    √    √
Starone    √    √    √    √
3    √    √    √    √
PSTN local    √    √    √    √
* PSTN long distance    √    √    √    √
** Call using voip 01016    √    √    √    √
*** International call using ‘+’ sign    √    √    √    √
*** International call using 008    √    √    √    √
*** International call using 001    √    √    √    √
Video test call                  
Indosat-mentari    √    √    √    √
Indosat-Im3 (510 01)    √    √    √    √
Indosat-Im3 (510 21)    √    √    √    √
Indosat-matrix    √    √    √    √
Telkomsel    √    √    √    √
Excelcomindo    √    √    √    √
3    √    √    √    √
SMS test                  
Indosat-mentari    √    √    √    √
Indosat-Im3 (510 01)    √    √    √    √
Indosat-Im3 (510 21)    √    √    √    √
Indosat-matrix    √    √    √    √
Fren    √    √    √    √
Telkomsel    √    √    √    √
Excelcomindo     √    √    √    √
Flexy    √    √    √    √
Starone    √    √    √    √
3    √    √    √    √
Emergency call                  
112    √    √    √    √
110    √    √    √    √
Data access                  
GPRS    √    √    √    √
3G    √    √    √    √
Internet access via HP    √    √    √    √
Video streaming                  
Video portal 136    √    √    √    √
wap.3g.indosat.com    √    √    √    √
Internet access via HP    √    √    √    √
I-ring                  
Registration    √    √    √    √
Change song    √    √    √    √
UMB                  
*1234#    √    √    √    √
                   
* Destination city will be included (more than 1 city, can be ask to regional people)   
** Destination city and the country will be included (more than 1 city, can be ask to regional)   
*** Destination country is included (more than 1 country)   
Destination number can be finding in last page


B.    GSM Interoperator Benchmarking Methodology
1.    General
a.    Team drive test should invite regional to supervise before benchmarking activity begins.
b.    Access permit to do drive test indoor will be issued by region
c.    Simcards used with the following configuration:
i.    MS1- Indosat-mentari
ii.    MS2 - Telkomsel-As
iii.    MS3 - Excelcomindo-bebas
d.    Drive test should be done using types of measurement listed as follows:
i.    Idle mode (mobile)
ii.    Qos mode (mobile)
iii.    PDD mode (stationary)
iv.    GPRS mode (stationary)
v.    SMS mode (stationary)
e.    The drive test benchmarking should be done using T610 mobile station on a PC running on tems investigation version 8.0 connected to a GPS for coordinate data collection.
f.    Only multipurpose vehicle will be used for benchmarking drive test
g.    The drive test routes will cover all cities and intercity roads which will be determined by indosat
h.    All of measurement shall use “in car” mobile station antennas
i.    All mobile station (hand phone) must located in the in the middle seat of vehicle, mobile station do not allow to put in the dashboard
j.    The benchmarking drive test should be performed between Monday and Saturday

2.    Coverage benchmarking (idle mode)
a.    Mobile station1 shall use Indosat-mentari simcards
b.    Mobile station 2 shall use Telkomsel-As simcards
c.    Mobile station 3 shall use Excelcomindo-bebas simcards
d.    Idle benchmarking test performed in mobile condition following routes
e.    The idle mode for each mobile station should be done without locking to any specific GSM band

3.    Voice benchmarking (Qos mode)
a.    Mobile station 1 shall use Indosat-mentari simcards and originate a call to test number 555
b.    Mobile station 2 shall use Telkomsel-As simcards and originate a call to test number 888
c.    Mobile station 3 shall 3 shall use Excelcomindo-bebas simcards and originate a call to test number 818
d.    The Qos measurement will have call sequence with 60 seconds connected and 10 seconds idle or wait time
e.    The benchmarking drive test should be performed during working hours from 08.00 to 20.00 on local time
f.    Qos benchmarking test performed in mobile condition following routes
g.    The Qos mode for each mobile station should be done without locking to any specific GSM band

4.    PDD benchmarking (PDD mode)
a.    Post dial delay (PDD) test should be done at a specific place with stationary mobile
Station in good radio condition for 35 times per simcard for 4 simcards:
i.    Indosat-Mentari
ii.    Indosat-Im3
iii.    Telkomsel-As
iv.    Excelcomindo-bebas

b.    All 4 simcards will have to call a specific destination as a summarized in the following table:
    Originating number
Destination number    Indosat
mentari    Indosat
Im3    Telkomsel
As    Excelcomindo
bebas
Indosat-mentari    √    √    √    √
Indosat-Im3    √    √    √    √
Telkomsel-As    √    √    √    √
Excelcomindo-bebas    √    √    √    √
PSTN local    √    √    √    √
PSTN long distance (Jakarta)    √    √    √    √
International call (Singapore) using ‘+’ sign    √    √    √    √

5.    GPRS benchmarking (GPRS mode)
Every mobile station should be connecting with dual com port data cable for this GPRS test. First com port will connect as MS and another com port will connect as DC cable.
GPRS test should be done at a specific place with stationary mobile station in good radio condition for 35 times per simcard for 4 simcards (Mentari, Im3, As, bebas):
The mobile stations will do following test:
•    Domestic Server
i.    http load: http://www.indosat.com
ii.    http load: http://www.telkomsel.com
iii.    http load: http://www.xl.co.id
•    Overseas Server
i.    http load: http://www.yahoo.com
ii.    http load: http://www.msn.com/
iii.    http load: http://www.bbc.co.uk/

GPRS HTTP Load will work on DC cable and the logic to create command sequence (*.csq) to GPRS HTTP Load is:
1)    Start
2)    Application testing-data service-connect-dial up (put phone number *99***1#)
3)    Application testing-data service-session-HTTP Load (put web page address)
4)    Application testing-data service-connect-hang up
5)    Repeat step number 2 to number 4 for 35 times
6)    End

a.    Ftp load a file
i.    Required test for ftp upload 200 KB file size on  http://www.drivehq.com/
ii.    Required test for ftp download 200 KB file size on  http://www.drivehq.com/

GPRS FTP files will work on DC cable and the logic to create command sequence (*.csq) to GPRS FTP is:
1)    Start
2)    Application testing-data service-connect-dial up (put phone number *99***1#)
3)    Application testing-data service-session-FTP-get/put (fill in the ftp address)
4)    Application testing-data service-connect-hang up
5)    repeat step number 2 to number 4 for 35 times
6)    End

b.    Ping a web site
•    Domestic Server
iv.    http load: http://www.indosat.com
v.    http load: http://www.telkomsel.com
vi.    http load: http://www.xl.co.id
•    Overseas Server
i.    http load: http://www.yahoo.com
ii.    http load: http://www.msn.com/
iii.    http load: http://www.bbc.co.uk/

GPRS ping test will work on DC cable and the logic to create command sequence (*.csq) to GPRS FTP is:
7)    Start
8)    Application testing-data service-connect-dial up (put phone number *99***1#)
9)    Application testing-data service-session-FTP-get (put web site address, number of ping is 35 )
10)    Application testing-data service-connect-hang up
11)    End

6.    SMS benchmarking (sms mode)
Every mobile station should be connecting with dual com port data cable for this SMS test. First com port will connect as MS and another com port will connect as DC cable.
SMS test should be done at a specific place with stationary mobile station in good radio
Condition for 35 times per simcard for 4 simcards:
The mobile stations will send and receive sms into its own simcards number with following test (on Net):
i.    Mobile station 1 Indosat-mentari simcard will send to its own
ii.    Mobile station 2 Telkomsel-As simcard will send to its own
iii.    Mobile station 3 Excelcomindo-bebas simcard will send to its own
iv.    Mobile station 4 Indosat-Im3 simcard will send to its own

The SMS command sequence (*.csq) will work on DC cable, and every send sms message should be distinguished by its contents E.g.:
i.    content of indosat-mentari : “send sms indosat-mentari1”, “send sms indosat-mentari2”, “send sms indosat-mentari3” etc
ii.    content of telkomsel-As : “send sms telkomsel-As1”, “send sms telkomsel-As 2”, “send sms telkomsel-As 3” etc
iii.    content of excelcomindo-bebas : “send sms excelcomindo-bebas 1”, “send sms excelcomindo-bebas2”, “send sms excelcomindo-bebas3” etc
iv.    content of indosat-Im3 : “send sms indosat-im31”, “send sms indosat-im32”, “send sms indosat-im3” etc

C.    KPI Benchmarking Formula
1.    Coverage benchmarking (idle mode)
Measurement is done in idle mode drive test. It was measured by extracting the Rx_level_sub (dBm) from the layer 3 message and then exporting it into MapInfo to calculate the signal strength for each mobile station

The valid value for Rx_level_sub (dBm) between -110 to -40 the other value of
Rx_level_sub (dBm) must be excluded from calculation.


2.    Voice benchmarking (Qos mode)
a.    CSSR (Call setup success rates)
Measurement is done in Qos mode drive test. Call setup will start when channel request message send by the mobile station and a successful call setup has been established when an alerting message being received by mobile station.
This information can be observed the layer 3 message.
Channel request is defined as a call attempt and alerting is defined as call setup in tems event definition. The CSSR formula is defined as following:
CSSR = ∑ Call setup / ∑ Call attempt

b.    CDR (Call drop rates)
Measurement is done in Qos mode drive test. Measured from call setup (alerting) until disconnect cause 16, a normal call clearing, known as normal call behaviors. A drop call will be pegged when there is no disconnect cause 16 received by mobile station and the mobile station goes to idle mode.
This is defined as call drop in tems investigation tems event definition. The CDR formula is defined as following:
CDR = ∑ Call drop / ∑ Call setup

c.    CSR (Call success rates)
Measurement is done in Qos mode drive test. It is derived from CSSR and CDR and CSR formula is defined as following:
CSR = [CSSR * (1-CDR)]

d.    SQI (Speech quality index)
Measurement is done in Qos mode drive test. It was measured by extracting the SQI from the layer 3 message and then exporting it into MapInfo to calculate the SQI for each mobile station.
The valid value for SQI between -20 and 30, The other value of SQI should be
excluded from calculation.

3.    PDD benchmarking (PDD mode)
PDD time is measured by taking the time between channel requests and alerting in layer 3 messages. The average time for PDD will be defined as below:
PDD time = Average [Δ Time (Alerting – channel request)]


4.    GPRS benchmarking (GPRS mode)
A.    Http Load a web page
•    Domestic Server
iv.    http load: http://www.indosat.com
v.    http load: http://www.telkomsel.com
vi.    http load: http://www.xl.co.id
•    Overseas Server
i.    http load: http://www.yahoo.com
ii.    http load: http://www.msn.com/
iii.    http load: http://www.bbc.co.uk/
a.    Formula for PDP context activate success rates:
∑ Activate PDP context Accept / ∑ Channel request
b.    PDP context activate time
Average [Δ Time (Activate PDP context Accept – channel request)]
c.    PDP context deactivate success rates
∑ Deactivate PDP context Accept / ∑ Channel request
d.    PDP context deactivate time
Average [Δ Time (Deactivate PDP context Accept – channel request)]
e.    Http load time
Average [Δ Time (Session end – Session start)]
f.    Current Coding scheme DL   percentage
Percentage of Current Coding scheme DL   1 (CS-1) = ∑ (CS-1) / ∑ (CS)
Percentage of Current Coding scheme DL   2 (CS-2) = ∑ (CS-2) / ∑ (CS)
Percentage of Current Coding scheme DL   3 (CS-3) = ∑ (CS-3) / ∑ (CS)
Percentage of Current Coding scheme DL   4 (CS-4) = ∑ (CS-4) / ∑ (CS)


B.    Ftp load a file
i.    Ftp upload 200 KB file size on  http://www.drivehq.com/
ii.    Ftp download 200 KB file size on  http://www.drivehq.com/
a.    PDP context activate success rates:
∑ Activate PDP context Accept / ∑ Channel request
b.    PDP context activate time
Average [Δ Time (Activate PDP context Accept – channel request)]
c.    PDP context deactivate success rates
∑ Deactivate PDP context Accept / ∑ Channel request
d.    PDP context deactivate time
Average [Δ Time (Deactivate PDP context Accept – channel request)]
e.    Ftp load time
Average [Δ Time (Session end – Session start)]
f.    Current Coding scheme DL   percentage
Percentage of Current Coding scheme DL   1 (CS-1) = ∑ (CS-1) / ∑ (CS)
Percentage of Current Coding scheme DL   2 (CS-2) = ∑ (CS-2) / ∑ (CS)
Percentage of Current Coding scheme DL   3 (CS-3) = ∑ (CS-3) / ∑ (CS)
Percentage of Current Coding scheme DL   4 (CS-4) = ∑ (CS-4) / ∑ (CS)

C.    Ping a web site
Are ping to the following address:
•    Domestic Server
iv.    http load: http://www.indosat.com
v.    http load: http://www.telkomsel.com
vi.    http load: http://www.xl.co.id
•    Overseas Server
i.    http load: http://www.yahoo.com
ii.    http load: http://www.msn.com/
iii.    http load: http://www.bbc.co.uk/

The quality indicator are :
a.    Ping response time
Average [Time (Event ping response delay)]
b.    Ping success rates
∑ Ping success / ∑ Ping request

5.    SMS benchmarking (SMS mode)
Measurement is done in sms mode in stationary test. And done on Net [Hand phone will act as a sender (A#) and a receiver (B#) of sms].
a.    Sms send success rates
Sms send start when channel request messages followed by CP-Data UL message send by the mobile station (A#) and successful delivered when a CP-Data  followed by channel release message being received by mobile station (B#).
This information can be observed in layer 3 message.

Sms send success rates = ∑ Channel release of B# / ∑ Channel request of A#



b.    Sms delivery time
SMS delivery time is measured by taking the time between channel request of A# and channel release of B# in layer 3 messages.
Sms send start when channel request messages followed by CP-Data UL message send by the mobile station (A#) and successful delivered when a CP-Data  followed by channel release message being received by mobile station (B#).

Average sms delivery time =
∑ Δ Time (Channel release of B# – Channel request A#) / ∑ Channel Request of A#

Country destination call
Number    Country    Phone number        Number    City    Phone number
1    AUSTRALIA    61 414 201 401        1    Bandung   
2    AUSTRIA    43 1 7983113        2    Jakarta   
3    BANGLADESH    880 2 98900 75        3    Bogor   
4    BELGIUM    32 2 2193532        4    Surabaya   
5    BRUNEI DS    673 2411188        5    Malang   
6    CANADA    16042610888        6    Etc   
7    CHINA    86 21 2535029        7    Etc . . .    
8    EGYPT    20 22 676600        8       
9    FRANCE    33 326056007        9       
10    GERMANY    49 69 90942148        10       
11    GREECE    30 210 330 1193        11       
12    HONGKONG    85224244386        12       
13    INDIA    91 11 2331 8383        13       
14    IRAN    98 213293500        14       
15    ITALY    39 0636892033         15       
16    JAPAN    81 3 33475515        16       
17    MACAO    8538913033        17       
18    MALAYSIA    60 12 2040 733        18       
19    NETHERLANDS    31 30 2386420        19       
20    NEW ZEALAND    64 9 8345342        20       
21    NORWAY    47 22775999        21       
22    OMAN    968 24633096        22       
23    PAKISTAN    92 51 2278799        23       
24    PHILIPINES    63 2 8172131        24       
25    PORTUGAL    351 917365562        25       
26    RUSIA    7 421 2213976        26       
27    SAUDI ARABIA    966 26472311        27       
28    SINGAPORE    6562255225        28       
29    SOUTH KOREA    82 513280083        29       
30    SPAIN    34 91 396 2679        30       
31    SRILANKA    94 11 2438458        31       
32    SWEDEN    46 8 108 210        32       
33    SWITZERLAND    41 1 2946700        33       
34    TAIWAN    886 7 3446612        34       
35    THAILAND    66 2 2342896        35       
36    TURKEY    90 212 5209898        36       
37    U.A.E    971 4 2022573        37       
38    UNITED KINGDOM    44 1344 602228        38       
39    U.S.A    1 303 3917777        39       
40    VIETNAM    84 88391 807        40