Check IOPS of your Oracle Database Storage for Cloud Migration

Method 1:

To do a quick check of Maximum IO per second (IOPS) of your underlying storage system use Oracle supplied IO Calibration tool. I believe this is available from Oracle 10g, but i have used it on Oracle 11g and it works well.
This tool will generate random single and large block reads to generate I/O and measure the IOPS and Mbps.
A very handy tool to estimate the required IOPS when migrating to cloud to build or select the proper storage on your cloud volumes.
Procedure to run the tool:
Note: Use this tool during a quiet time or quiese mode, so no other connections are made to database other than your sqlplus session.
This will give near accurate results.

Login as sysdba to server sqlplus session. Make a local connection.
Run below:
SQL> SET SERVEROUTPUT ON
SQL> DECLARE
l_latency PLS_INTEGER;
l_iops PLS_INTEGER;
l_mbps PLS_INTEGER;
BEGIN
DBMS_RESOURCE_MANAGER.calibrate_io (num_physical_disks => 1,
max_latency => 20,
max_iops => l_iops,
max_mbps => l_mbps,
actual_latency => l_latency);
DBMS_OUTPUT.put_line(‘Max IOPS = ‘ || l_iops);
DBMS_OUTPUT.put_line(‘Max MBPS = ‘ || l_mbps);
DBMS_OUTPUT.put_line(‘Latency = ‘ || l_latency);
END;
/

Note: The above script will take 2 inputs,
Max Latency – in milli seconds, select the acceptable latency in block reads (default 20 milli seconds is good for most storage types)
Num_Physical_disks – Set it to 1, if you have datafiles spread across multiple physical disks (like in ASM) select the correct number of disks attached to your +Data disk group, otherwise input 1.

This will run for few minutes, leave the session to run.

You can monitor the progress by querying V$IO_CALIBRATION_STATUS.

SQL> select * from V$IO_CALIBRATION_STATUS;

Once the IO calibration is completed, it will give you the results as below:

Max IOPS = 667
Max MBPS = 179
Latency = 21

The above information is also recorded in a table DBA_RSRC_IO_CALIBRATE.

Using the max iops and max mbps you may select the right storage and instance type.
Example: To support 667*10%* around 700 IOPS you may select a GP2 SSD type storage in AWS with 250 GB (it gives approximately 750 IOPS) or you can lock down 250 GB IO storage with 1000 PIOPS.
* To cover for any unforeseen spikes, it would be a good idea to add an additional 10 percent to this peak IOPS number to account for the actual IOPS needed for your database. Also you may add for writes as well.

Method 2:

Another handy method to calculate the IOPS is by using below script and schedule to run it every day or multiple times in a day for a period and check the results.

-Modify the values of run_duration and capture_gap to best fit your needs

– First create a table to store I/O results
CREATE TABLE peak_iops_measurement (capture_timestamp date,total_read_io number, total_write_io number, total_io number,total_read_bytes number, total_write_bytes number, total_bytes number);

– Schedule below script using shell or run it few times on the database as sysdba

The script runs for 1 hour (3600 secs) and captures data every 5 minutes.

DECLARE
run_duration number := 3600;
capture_gap number := 5;
loop_count number :=run_duration/capture_gap;
rdio number;
wtio number;
prev_rdio number :=0;
Prev_wtio number :=0;
rdbt number;
wtbt number;
prev_rdbt number;
Prev_wtbt number;
BEGIN
FOR i in 1..loop_count LOOP
SELECT SUM(value) INTO rdio from gv$sysstat WHERE name =’physical read total IO requests’;
SELECT SUM(value) INTO wtio from gv$sysstat WHERE name =’physical write total IO requests’;
SELECT SUM(value)* 0.000008 INTO rdbt from gv$sysstat WHERE name =’physical read total bytes’;
SELECT SUM(value)* 0.000008 INTO wtbt from gv$sysstat WHERE name =’physical write total bytes’;
IF i > 1 THEN
INSERT
INTO PEAK_IOPS_MEASUREMENT
(
CAPTURE_TIMESTAMP,
TOTAL_READ_IO,
TOTAL_WRITE_IO,
TOTAL_IO,
TOTAL_READ_BYTES,
TOTAL_WRITE_BYTES,
TOTAL_BYTES
)
VALUES
(
SYSDATE,
( RDIO-PREV_RDIO)/5,
(WTIO-PREV_WTIO)/5,
((RDIO-PREV_RDIO)/5)+((WTIO-PREV_WTIO))/5,
(RDBT-PREV_RDBT)/5,
(WTBT-PREV_WTBT) /5,
((RDBT-PREV_RDBT)/5)+((WTBT-PREV_WTBT))/5);

END IF;

prev_rdio := rdio;
prev_wtio := wtio;
prev_rdbt := rdbt;
prev_wtbt := wtbt;

DBMS_LOCK.SLEEP(capture_gap);
END LOOP;
COMMIT;
EXCEPTION
WHEN OTHERS THEN
ROLLBACK;
END;
/

– Check the IOPS with below query
SELECT MAX(total_io) PeakIOPS FROM peak_iops_measurement;

Using the max iops and max mbps you may select the right storage and instance type.
Example: To support 667*10%* around 700 IOPS you may select a GP2 SSD type storage in AWS with 250 GB (it gives approximately 750 IOPS) or you can lock down 250 GB IO storage with 1000 PIOPS.
* To cover for any unforeseen spikes, it would be a good idea to add an additional 10 percent to this peak IOPS number to account for the actual IOPS needed for your database. Also you may add for writes as well.

 

 

Inmemory initialization parmaeters in database 12c 12.1.0.2

Make sure to disable inmemory features in 12c database if you do not have oracle licenses for that feature or avoid to use the feature accidentally.

alter system set INMEMORY_SIZE =0;

The default value is 0, which means that the IM column store is not used.

ALTER SYSTEM set INMEMORY_QUERY = DISABLE;

Set this parameter to DISABLE to disable in-memory queries.

ALTER SYSTEM set INMEMORY_CLAUSE_DEFAULT = ‘NO INMEMORY’; or

ALTER SYSTEM set INMEMORY_CLAUSE_DEFAULT =”;

If the INMEMORY_CLAUSE_DEFAULT parameter is unset or set to an empty string (the default), only tables and materialized views explicitly specified as INMEMORY will be populated into the IM column store. Setting the value of the INMEMORY_CLAUSE_DEFAULT parameter to NO INMEMORY has the same effect as setting it to the default value.

alter system set INMEMORY_FORCE=OFF;

If OFF is specified, then even if the IM column store is configured on this instance, no tables or materialized are populated in memory.

ALTER SYSTEM set INMEMORY_MAX_POPULATE_SERVERS= 0;

This parameter has meaning only if the INMEMORY_SIZE parameter is also set to a positive value. You can also set this parameter to 0 to temporarily disable populate tasks on the system from executing.

ALTER SYSTEM set INMEMORY_TRICKLE_REPOPULATE_SERVERS_PERCENT = 0;

This parameter has meaning only if the INMEMORY_SIZE parameter is also set to a positive value.

If you want to disable trickle repopulate, this parameter can be set to 0.

ALTER SYSTEM/SESSION set OPTIMIZER_INMEMORY_AWARE = FALSE;

Setting the parameter to false causes the optimizer to ignore the in-memory property of tables during the optimization of SQL statements.

Demantra Engine Fails

Provider=MSDAORA, Database Error [db_win_cpp,135], Source: ADODB.Connection, Description: Provider cannot be found. It may not be properly installed.
image001

Solution for above error:

When you run the Engine, source oracle client binaries from 11.2.0.1 software. I have faced this issue when i was using 11.2.0.3 client.

set oracle_home=D:\app\oracle32\product\11.2.0.1\client_1

set path=%oracle_home%\bin;%path%

cd “Root\Oracle Demantra Spectrum\Demand Planner\Analytical Engines\bin”
 
EngineManager.exe EBSTEST demantra <demantra_password> 1 1

 

Runs successfully..

VMWare – VMotion Best Practices for Oracle Instances

I would like to list the best practices, i came across Oracle articles and VMWare white papers.

  • All ESX server host hardware, in particular the CPU, must be compatible
  • All virtual switches must be configured the same way for all participating ESX server hosts
  • Use a separate non-routable subnet for all Vmotion traffic or dedicated NICs for service console and Vmotion
  • Run a private Gigabit Ethernet migration network between all Vmotion enabled managed hosts
  • Incase a LUN has to be shared between multiple VMs, set “DisallowSnapshotLUN” value to 0 in virtual center
  • There needs to be free memory greater than the VM NVRAM file size
  • Make sure that Vmotion has unique IP and there aren’t duplicate Vmotion IPs
  • Make sure that Vmotion vmkernel portgroup is on its own vswitch and has a unique VLAN ID if VLANs are being used
    • VMware Network Documents also state that Vmotion and iSCSI vmkernel interfaces need to be on isolated networks vmkernel PGs in separate vswitches, and on separate networks or VLANs
  • Make sure that Vmotion is NOT enabled on any other portgroups other than vmkernel interface intended for Vmotion
  • Check time settings, enable NTP for the ESX/ESXi Servers in the clusters
  • VMware suggests where possible to not route Vmotion traffic to limit the number of hops that Vmotion has to take, and only one vmkernel interface is permitted per vswitch
    • Every hop that Vmotion traverses adds to Vmotion latency. This is not always possible but it is a VMware suggestion
  • Follow the troubleshooting guidelines from ref. notes below:
    • Troubleshooting Vmotion Failures and Vmotion Configuration Best Practices (Doc ID 1518833.1)

SHA2 certificates with EBS 12.1.3

 

Outbound Encryption involves connections from Oracle E-Business Suite to external site(s). For outbound connections, the SHA (can be SHA-1 or SHA-2) signed PKI certificate is requested from a CA by a site you are connecting to from Oracle E-Business Suite is certified.
For this case, Oracle E-Business Suite is acting as an HTTPS client. You must trust the root CA of the remote server’s certificate chain in your truststore. Example include, but are not limited to the following:

  • Punchout in iProcurement.
  • XML Gateway connection to a partner applications.
  • Payments credit card processing.
  • Dunn & Bradstreet (HZ).
  • International Trade Management (ITM) for screening orders and deliveries.
  • CIS Tax Module

Outbound encryption for iProcurement and XML gateway to use SSLv3 with TLS / SHA2 certificates:

  • Release 12.1: Apply Patch 19835592:R12.ICX.B “Fix for Bug 19835592“
    If the supplier punchout site supports both SSLv3 and TLS, or TLS only, then it will work after applying the patch.

    • Any punchout suppliers who are only using SSLv3 will need to migrate to (or add) TLS protocol. The SSL protocol (v2 or v3) is no longer supported for use with Oracle iProcurement. Supplier sites will need to use TLS protocol
    • The fix also supports any TLS v1 version (TLS v1.0, v1.1 and v1.2)
  • For XML gateway
    • Follow the instructions in the patch README and apply the following patch: 19909850

SHA-2 signed PKI certificates are now certified for inbound connections to the Oracle HTTP Server (OHS) delivered with Oracle E-Business Suite 12.1.3

You must apply the minimum requirements when using SHA-2 signed PKI certificates. Minimum requirements include the following:

  • Upgrade FMW 10.1.3 to 10.1.3.5
  • Apply at least the October 2015 CPU to FMW 10.1.3.5
  • Follow instructions from below document for requesting SSL certificates, and loading into Oracle Wallet:
    • Enabling SSL or TLS in Oracle E-Business Suite Release 12 (Doc ID 376700.1)

Oracle Database Security Feature in 11g “REDACTION”

Oracle has introduced redaction feature in database 12c and added later to 11g in 11.2.0.4 patchset.

Redaction feature is available with dbms_redact.

Some examples:

a. If you want to redact credit card data in table:

Define a policy with functions of your choice. You can use FULL redaction or Partial or Regular Expression mode. This example is for partial. You should know the max length of the column for correct input/output parameters.

select max(length(COLNAME)) from <schema>.<table_name>;

Now, lets check the userenv for session_user.

SELECT SYS_CONTEXT (‘userenv’, ‘SESSION_USER’)    FROM DUAL;

So, go ahead and define a policy for a table which has max length of 19 char and column defined as varchar..

begin
dbms_redact.add_policy(
object_schema => ‘SCOTT’,
object_name => ‘TABLENAME’,
column_name => ‘COLNAME’,
policy_name => ‘CREDITCARD_MASKIT’,
function_type => DBMS_REDACT.PARTIAL,
function_parameters => ‘VVVVVVVVVVVVVVVVVVV,VVVVVVVVVVVVVVVVVVV,*,1,10’,
expression => ‘SYS_CONTEXT(”USERENV”,”SESSION_USER”)
= ”SCOTT”’);
end;

 

Check it:

select colname from scott.tablename;

You will get out put as * for first 10 digits and after 10 digits the orig. values of data is displayed.

**********714894028

—————————————————————————————————————–

b. If you want to redact a number with length as 10.

I want to write a policy, the first 5 numbers will be replaced with digit 9 and rest all ignored in my select query.

begin
dbms_redact.add_policy(
object_schema => ‘TEST’,
object_name => ‘TABLEA’,
column_name => ‘NUMBERA’,
policy_name => ‘MASK_PARTIAL_A’,
function_type => DBMS_REDACT.PARTIAL,
function_parameters => ‘9,1,5’,
expression => ‘SYS_CONTEXT(”USERENV”,”SESSION_USER”)
= ”TEST”’);
end;
/

You can query below dictionary views to see the redact column and function defined in a database.

select * from redaction_policies;

select * from redaction_columns;

——————————————————————————————-

You can alter, drop policies.

begin

DBMS_REDACT.DROP_POLICY (
object_schema => ‘TEST’,
object_name => ‘TABLEA’,
policy_name => ‘MASK_PARTIAL_A’);

end;

References:

Some examples of implementing Data Redaction using DBMS_REDACT (Doc ID 1588270.1)
sys_context: http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions165.htm
dbms_redact: http://docs.oracle.com/cd/E11882_01/appdev.112/e40758/d_redact.htm#ARPLS73812

Failed to navigate from Registry/WizardState: Error while initializing registry. See logs for more details

When you install older version of Hyperion EPM 11.1.2.1 you may face the below error while configuring shared services:

Failed to navigate from Registry/WizardState: Error while initializing registry. See logs for more details

Three things that needs to check are:

a. Valid Hostname for the server

–check your /etc/hosts on linux or hosts file on Windows

b. Database User Privileges

— Use below statement to add necessary basic privileges to the db user

Launch SQLPlus on the Oracle database server or on a machine with Oracle client installed
Login as system
Type the following command:
Grant Connect, Resource, Create View, Create Session, unlimited tablespace to HssDBUser;
Commit;

c. Go to folder Oracle\Middleware\user_projects\epmsystem1\config\foundation\11.1.2.0\product\11.1.2.0 and edit the foundation_1_config.xml. Set the following parameter values from configured or failed to Pending:

applicationserverdeployment
FndCommonSetting
relationalStorageConfiguration
webServerConfiguration
Restart the configuration utility and reconfigure again.

The above step is optional, it wasn’t necessary for me as there were no application folders in user_projects.