OGG中discard文件压缩数据与insert操作关系问题

Wizard

请问OGG中，当replicat同步时如果发现update操作找不到要更新的数据，是不是，就会向discard文件中插入一条记录，同时报no data found ， mapping error 然后abending？
我的replicate配置如下：
     REPLICAT RMSS03p
     ASSUMETARGETDEFS
     APPLYNOOPUPDATES
     TARGETDB XXXXXX, USERID XXXXXXX, PASSWORD XXXXXXX
     DISCARDFILE ./dirrpt/RMSS03p.DSC, PURGE
      --HANDLECOLLISIONS
      GROUPTRANSOPS 10000
      MAXTRANSOPS 10000
      MAP dbo.Product, TARGET dbo.Product, KEYCOLS (ProductCode) ;

abending后我查看discard file，其现实内容如下：
     Current time: 2012-06-22 10:29:55
Discarded record from action ABEND on error 100

No data found
Aborting transaction on ./dirdat/03/yp beginning at seqno 37 rba 1605919
                         error at seqno 37 rba 2385615
Problem replicating DBO.PRODUCT to DBO.PRODUCT
Record not found
Mapping problem with compressed key update record (target format)...
*
ProductCode = <Raw Data>
000000: 00 00 00 00 43 00 41 00 42 00 30 00 31 00 30 00 |....C.A.B.0.1.0.|
000010: 30 00 39 00                                     |0.9.            |
Category = <Raw Data>
000000: 00 00 00 00 43 00 41 00 42 00                   |....C.A.B.      |
Description = <Raw Data>
000000: 00 00 00 00 41 00 2d 00 50 00 6f 00 77 00 65 00 |....A.-.P.o.w.e.|
000010: 72 00 20 00 34 00 20 00 50 00 69 00 6e 00 20 00 |r. .4. .P.i.n. .|
000020: 50 00 6f 00 77 00 65 00 72 00 20 00 74 00 6f 00 |P.o.w.e.r. .t.o.|
000030: 20 00 32 00 78 00 53 00 41 00 54 00 41 00 20 00 | .2.x.S.A.T.A. .|
000040: 50 00 6f 00 77 00 65 00 72 00 20 00 53 00 70 00 |P.o.w.e.r. .S.p.|
000050: 6c 00 69 00 74 00 74 00 65 00 72 00 20 00 43 00 |l.i.t.t.e.r. .C.|
000060: 61 00 62 00                                     |a.b.            |
SalesPrice1 = -2.47250199712742e+084
SalesPrice2 = 0
SalesPrice3 = 0
SalesPrice4 = 0
LimitStockQty = 0
GSTRate = 0
LastOrderPrice = -9.25596411572893e+061
Live =
000000: 00                                              |.               |
Barcode = <Raw Data>
000000: 00 00 00 00 43 00 42 00 42 00 2d 00             |....C.B.B.-.    |
StockControl =
000000: 00                                              |.               |
GSTStatus =
000000: 00                                              |.               |
Notes = <Raw Data>

Description1 = <Raw Data>

Description2 = <Raw Data>

Description3 = <Raw Data>

Memorandum = <Raw Data>

StockOnOrder = 0
StockCommitted = 0
SerialID =
000000: 00 00 00 00                                     |....            |
SpecialDate1 =
000000: 00 00 00 00 31 38 39 39 2d 31 32 2d 33 30 20 30 |....1899-12-30 0|
000010: 30 3a 30 30 3a 30 30                            |0:00:00         |
SpecialDate2 =
000000: 00 00 00 00 31 38 39 39 2d 31 32 2d 33 30 20 30 |....1899-12-30 0|
000010: 30 3a 30 30 3a 30 30                            |0:00:00         |
SpecialPrice = 0
ImageLoadPath = <Raw Data>
---------------------------------------------------------------------------------------------------------------------------
请问：我查看trail文件，其记录也是以这种方式记录的，是不是ogg会对记录进行压缩，这个不是很好识别出错的记录信息，能不能设置ogg，以非压缩方式记录，方便我定位出问题的记录？

ShineCQY

应该是两边表不一致造成的，更新的时候发现目标端表中没有此行数据。 可以使用select count(*) from talbe_name;简单看下两边的表是否数据一致。还可以打开trail 文件，根据RBA号定位到要更新的是那些记录.

Maclean Liu(刘相兵

discard file record for example:

1. Current time: 2011-03-17 15:51:01
   
2. Discarded record from action ABEND on error 1400
   
3. 
   
4. OCI Error ORA-1400: cannot insert NULL into
   
5. ("ETRADE"."ETJP_NEWSHEADLINE"."PNAC") (status = 1400),
   
6. SQL <INSERT INTO "ETRADE"."ETJP_NEWSHEADLINE"
   
7. ("SEQ_NO","MESSAGE_TYPE","PROC_DATE","TAKE_TIME","CATEGORIES","PRODUCTS","BCAS
   
8. T_TEXT","UPDATE_DT") VALUES (:a0,:a1,:a2,:a3,:a4,:a5,:a6,:a7)>
   
9. 
   
10. Aborting transaction on /opt/ogg/dirdat/news/re beginning at seqno 0 rba 2523
    
11.                          error at seqno 0 rba 2523
    
12. Problem replicating ETRADE.ETJP_NEWSHEADLINE to ETRADE.ETJP_NEWSHEADLINE
    
13. Mapping problem with compressed key update record (target format)...
    
14. *
    
15. SEQ_NO = 201103171444540001
    
16. SEQ_NO = 201103171444540002
    
17. MESSAGE_TYPE = 2
    
18. PROC_DATE = 20110317
    
19. TAKE_TIME = 144454
    
20. CATEGORIES = FEX HDN MK2
    
21. PRODUCTS = NULL
    
22. BCAST_TEXT = <Raw Data>
    
23. 000000: a1 cc b3 b0 b0 d9 a5 de a1 bc a5 b1 a5 c3 a5 c8 |................|
    
24. 000010: a5 a2 a5 a4 a1 cd a5 c9 a5 eb 37 39 2e 34 30 b1 |..........79.40.|
    
25. 000020: df c9 d5 b6 e1 a1 a2 38 30 b1 df c9 d5 b6 e1 a4 |.......80.......|
    
26. 000030: ce be e5 c3 cd a4 ce bd c5 a4 b5 b0 d5 bc b1    |............... |
    
27. UPDATE_DT = 2011-03-17 14:55:00
    
28. ====

复制代码

关于discard file的解释可以参考

How to Read an Oracle GoldenGate (OGG) Discard file - DISCARDFILE

Applies to:
Oracle GoldenGate - Version: 10.0.0.0 and later   [Release: 10.0.0 and later ]
Information in this document applies to any platform.
Abstract
The OGG Discard file (DISCARDFILE) contains a great deal of information useful to resolution of OGG issues. This paper documents how to read and use the contents of the discard file.
Document History
Author: Michael Zerger
Create Date 22-Mar-2011
Update Date 22-Mar-2011
Expire Date 22-Mar-2016 (ignore after this date)
How to Read an Oracle GoldenGate (OGG) Discard file - DISCARDFILE
Overview:

The discard file is very useful in identifying OGG problems and customer data issues. This paper highlights important features and uses of the discard file. This currently refers to discard files produced in v10.4 and v11.x versions of OGG. Earlier versions of OGG contain the same kind of information but less of it.

The Discard File:
OGG provides the option to have a discard file written when an object (extract or replicat) starts. The file is optional but should be considered required for replicats.



Summary
The discard file is key to resolving customer data issues and to understanding what OGG thinks the issue is with the data. It must be understood to be useful. All discard files of current releases (v10.4x and up) contain a header written as soon as the discard file is opened. The file is opened at the location specified in the discardfile option of the extract or replicat parameter file.

The header looks like this:

GoldenGate Delivery for Teradata process started, group R15198 discard file opened: 2009-11-05 10:14:54

Current time: 2009-11-05 10:14:55

No data found

These few lines have important information. This report shows that this is for a replicat (delivery) process named r15198. It was opened at  2009-11-05 10:14:54 and ran until 2009-11-05 10:14:55, This shows us that there is no abend recorded and nothing was discarded during this run. The timestamp can be correlated to the start / stop time of the replicat so that we know this is the correct file. In this case, the process ran for about one second so it was likely stopped. This is also useful information. The fact that the header is written also tells us that this discard file was not written by older OGG code which did not create headers.

The discard file will show discarded transactions. It will also show records discarded as part of a transaction in which one row caused the discard of the transaction and all other rows of the transaction were consequently discarded. It is there fore important to note which row contains the actual error. Here is an example:

Discarded record from action ABEND on error -2971

[SQL error -2971][NCR][ODBC Teradata Driver][Teradata Database] The AMP Lock table has overflowed.

Aborting transaction on e:\gg\teradata\dirdat\cf beginning at seqno 0 rba 0
                         error at seqno 0 rba 4293078

It can be seen that the transaction starts at sequence (trail file) number 0, rba 0.  The whole transaction starting here will be thrown away and all of its rows will be rolled back from the target if they have been applied. The row with the error starts at RBA 4293078, several rows into the trail. The error is caused by a database problem, not directly by the replicat.

The discard file will show the error that caused the discard.


Operation failed at seqno 0 rba 0
Discarding record on action DISCARD on error 100
Problem replicating PASSPORT.TIDPOSCD to MZERGER.PP_TIDPOSCD
Record not found
Error (100) occurred with compressed key update record (target format)...
*
PURCHASE_ORDER_NBR = 00247828
PO_REVISION_NBR = 001
PASSPORT = POACK   
DATE_SCD = 20090605
GEN_ARG = 10000
TIME_STAMP = 2009-06-05-05.06.06.000013

This is the data as it will be applied. It is sometimes obvious that data is malformed. Other times, the DDL must be examined to determine the key or index from the data shown by the discard file. The user should then perform an SQL lookup of the row as defined by the key values. This will show whether the replicat is correctly reporting an error. It does not generally show the causes of the error.

In this particular case, the error is shown as:
Error (100) occurred with compressed key update record (target format)

This is a common error and indicates the incoming data, which is compressed, does not contain sufficient information to create the target key or index.

Sometimes the error is created somewhere other than in the target database contents. In this example:

Current time: 2010-02-18 12:23:07
Discarded record from action ABEND on error -6706

[SQL error -6706][NCR][ODBC Teradata Driver][Teradata Database] The string contains an untranslatable character.


Aborting transaction on e:\gg\teradata\dirdat\ab beginning at seqno 0 rba 179
                         error at seqno 0 rba 179

the error is caused by an untranslatable character found in the data. The row to be updated was found but the database would not allow the data to be applied.

Replicat parameters can also set discard error codes and the occur in the discard file.

In this example:

Operation failed at seqno 0 rba 1245
Discarding record on action DISCARD on error 21000
Problem replicating MZERGER.TAB8811S to MZERGER.TAB8811T
Filter not passed for compressed update record (source format)...
*
CUST_CODE = JANE
NAME = ROCKY FLYER INC.
CITY = DENVER
STATE = CO

replicat reports that there was an error 21000 encountered and that abended replication. The error was set in the replicat with this statement:

TARGET mzerger.tab8811t,
filter (
on update,
(
((1 = @streq (before.name, name))   )       AND
((1 = @streq (before.city, city))   )
),            
raiseerror 21000
)
;

combined with the error option that causes the discard.
reperror 21000, discard

Discard files can be set to purge or append on restart. Appended discard files are difficult to read but give a history of errors. If the file fills up, it is likely that there are serious replication or data issues.

For debugging, it is useful to rename the discard file t preserve is for research and then set the two parameters (replicat only)

grouptransops 1
maxtransops 1

and restart the replicat. The replicat will abend exactly on the row causing the transaction and give very good report and discard information. The replicat should not be allowed to run with these parameters except in error recovery and debug modes.

For recovery purposes, it is recommended that exception tables be used instead of discard files.

Further information on reading extract specific discard files is available at: